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ABSTRACT 
The  evaluations  mode  by  36  Navy  fighter  pilots  of  64  air-to-air 
combat  situations  are  statistically  analyzed  to  detect  any  significant 
differences  between  pilots'  evaluative  techniques  in  relation  to  their 
rank,  flight  hours  and  combat  missions.     Predictor  equations  are  computed 
and  used  in  a  self-analyzing,  self -modifying  artificial  intelligence 
program  modeled  on  an  instructor-flight  student  interactive  situation. 
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L.    INTRODUCTION 

From  the  legendary  Baron  von  Richtofen  to  the  popular  Snoopy- 
flying  his  doghouse  through  the  comic  strips,  the  image  of  the  fighter 
pilot  is  one  of  an  enigma  capable  of  performing  complex  and  dangerous 
tasks  in  an  environment  where  fractions  of  a  second  can  spell  the  dif- 
ference between  success  and  filure,   life  and  death.    This  fascinating 
picture  of  a  highly  trained,  assured  and  swaggering  superbeing  beckons 
to  be  studied;  to  have  the  mystique  lifted.     Hence,  the  impetus  of  this 
research:    the  fighter  pilot. 

Other  than  the  satisfaction  of  human  curiosity,  the  study  of  the 
fighter  pilot  -in  this  case  the  Naval  fighter  pilot-  can  ultimately  result 
in  improved  training  and  selection  of  future  pilots  by  understanding  the 
activities  of  successful  pilots;  improved  combat  techniques  by  adaptation 
of  the  successful  pilots'  techniques;  improved  surface-to-air  and  air-to- 
air  communications  through  better  understanding;  and  improved  electronic 
and  mechanical  pilot  aids  to  enhance  pilot  performance. 

A.         OBJECTIVES 

The  objectives  of  this  research  lay  in  two  major  areas.    The  first 
was  to  address  the  questions  of  how  Navy  fighter  pilots  evaluate  an  air- 
to-air  combat  threat,  i.e.  ,   pilot  versus  pilot  situations.     How  does  the 
pilot  reach  his  decision?     Can  this  decision  be  somehow  quantified  for 


computer  applications?    What  differences  exist  among  pilots  in  their 
evaluations  of  air-to-air  threat? 

The  second  major  area  lies  in  the  field  of  artificial  intelligence. 
Classically  concerned  with  games  such  as  chess  and  checkers  with  well 
defined  rules  for  determining  success  and  failure  [1,  2],  artificial  intel- 
ligence has  seen  little  application  in  real  life  "games."    A  notable 
exception  is  Clarkson's  portfolio  program  which  has  reached  partial 
agreement  with  a  successful  bank  trust  administrator  [3]  .     Heuristics, 
such  as  the  minimax  technique  whereby  a  player  will  always  attempt  to 
minimize  his  maximum  loss  [4],  and  binary  choice  selection  [5],  although 
fine  for  board  games,  find  little  application  in  games  of  life.    As  stated 
by  Rigney  [6]: 

"For  decision  tasks  in  which  the  outcome  is  of  great  consequence 
and  the  time  for  decision  is  short,  as  in  many  military,  business, 
and  medical  decisions,  the  strategy  the  decision  maker  uses  is 
of  special  interest." 

From  the  strategies  hopefully  revealed  in  the  attainment  of  the  first 
objective,  an  artificial  intelligence  application  is  sought.    As  brought 
out  by  Shepard  [7],  an  obvious  disparity  exists  between  "the  effortless- 
ness and  surety  of  most  perceptual  decisions  and  the  painful  hesitation 
and  doubt  characteristic  of  these  'higher  level'  decisions"  and  strategies. 
To  overcome  this  disparity  in  a  "game"  of  ill  defined  rules  and  high  stakes 
is  the  objective  of  the  artificial  intelligence  application. 


B..         DEFINITIONS 

At  this  time  some  definitions  of  terms  used  throughout  this  research 
should  be  brought  forward.    Artificial  intelligence  as  used  in  this  study 
encompasses  the  construction  of  "computer  programs  which  exhibit 
behavior  that  we  call  'intelligent  behavior'  when  we  observe  it  in  human 
beings  [8]." 

The  area  of  artificial  intelligence  most  apt  for  this  study  is 
decision  making  in  game  playing.    Hence  the  game  is  defined  as  air- 
to-air  combat  as  experienced  in  the  Vietnam  area  of  operations. 

C.        ASSUMPTIONS 

Certain  assumptions  used  throughout  the  study  are: 

(1)  Data  such  as  military  rank  and  threat  evaluation  decisions 
can  be  placed  on  a  quantifiable  continuum. 

(2)  Since  no  criteria  of  deciding  success  or  failure  in  the  game 
exist  short  of  actual  combat  results,  expertise  in  the  game 
will  decide  successes  or  losses. 

(3)  Expertise  is  directly  proportional  to  military  rank,  flight 
hours  and  total  number  of  combat  missions. 

(4)  The  data  base  used  is  a  representative  sample  of  Navy 
fighter  pilots  . 


HE..    EXPERIMENTAL  PROCEDURE-DETERMINING  DECISION  STRATEGY 

In  an. attempt. to  isolate  and  identify  the  important  variables  con- 
sidered by  a  fighter  pilot  in  his  evaluation  of  an  air-to-air  combat 
situation,  a  mail-out  survey  was  designed.     In  the  survey  the  subjects 
were  asked  to  assume  the  following  hypothetical  situation: 

(1)  The  subject  is  an  airborne  Target  Area  Combat  Air  Patrol 
(TARCAP)  over  enemy  territory  in  Southeast  Asia. 

(2)  The  subject's  aircraft  is  configured  and  armed  for  air-to-air 
combat  only;  hence,  a  fighter. 

(3)  The  subject's  primary  mission  is  to  intercept  and  destroy 
enemy  air  raids  threatening  friendly  airborne  strike  forces. 

(4)  The  TARCAP  and  immediate  strike  group  is  the  only  friendly 
force  in  the  area.    Therefore,  the  subject  is  free  to  fire  at 
any  threat  without  positive  visual  identification. 

Eleven  different  items  of  additional  information  were  then  presented 

to  the  subject.    These  were  presented  in  pairs  yielding  a  total  of  55 

combinations.     The  items  of  information  were: 

-    (1)        Relative  position  of  enemy  aircraft  from  the  friendly  strike 
group . 

(2)  Type  of  enemy  aircraft. 

(3)  Number  of  enemy  aircraft. 

(4)  Number  of  friendly  aircraft. 

(5)  Range  from  friendly  aircraft  to  enemy  aircraft. 

(6)  Speed  of  enemy  aircraft. 

(7)  Friendly  aircraft  capabilities. 
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(8)  Altitude  of  enemy  forces  . 

(9)  Anti-aircraft  activity  in  the  area. 

(10)        Bearing  of  enemy  from  friendly  forces  . 

(11         Closure  rate  of  friendly  and  enemy  forces. 
The  subject  then  had  to  choose  which  one  of  the  pieces  of  infor- 
mation presented  in  each  pair  would  be  of  greater  value  in  evaluating 
the  threat  of,  and  in  planning  engagement  tactics  against,  enemy  air 
units.    An  additional  alternative  was  given  whereby  each  member  of  the 
pair  could  be  evaluated  as  being  egual  in  importance.    Appendix  A  con- 
tains a  sample  of  this  questionnaire.     Ninety-seven  Naval  aviators 
responded  to  this  questionnaire  of  which  68  or  70.1%  were  pilots  and 
2  9  or  2  9.9%  were  Radar  Intercept  Officers,  the  second  crewman  in  the 
F-4  Phantom  jet.     Eleven  or  11.3%  were  F-8  Crusader  pilots,  the  rest 
of  the  pilots  being  F-4  pilots.     Sixty-three  or  65.0%  of  the  aviators  had 
combat  experience  over  Korea  or  Vietnam.     Nine  or  9.4%  had  engaged  in 
actual  air-to-air  combat. 

From  a  scaling  of  the  variables  from  the  pair-comparisons  data 
the  variables  were  ranked  as  follows   (in  descending  order  of  importance) 

(1)  Relative  position  of  enemy  aircraft. 

(2)  Bearing  of  enemy. 

(3)  Range  of  enemy. 

(4)  Altitude  of  enemy. 

(5)  Friendly  aircraft  capabilities  . 
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(6)  Number  of:  enemy. 

(7)  Rate  of  closure  . 

(8)  Anti-aircraft  activity. 

(9)  Type  of  enemy  aircraft. 

(10)  Speed  of  enemy. 

(1 1)  Number  of  friendly  . 

From  the  results  of  the  first  survey,  and  with  the  consideration 
of  the  type  of  information  available  to  the  pilot  in  the  cockpit,  a  second 
mail-out  survey  was  designed.    The  variables  considered  in  this  survey 
were  limited  to  six  in  number.    Fuel  states  and  rules  of  engagement  were 
added  for  realism.    The  remaining  four  variables  were  considered  as  being 
information  a  pilot  would  have  at  his  access  from  either  his  cockpit 
instruments  or  via  a  surface  or  airborne  controller.    The  variables  were 
limited  to  two  states,  hence  25  or  64  possible  combinations  were  con- 
sidered by  each  subject.    The  variables  and  their  states  were: 

(1)  Fuel  above  bingo;  the  maximum  amount  of  fuel  that  can  be 
utilized  before  fuel  level  drops  below  that  necessary  for  a 
safe  return  to  home  base. 

(a)  1000  pounds. 

(b)  2  500  pounds. 

(2)  Rules  of  engagement;  when  the  subject  may  take  the  suspected 
aircraft  under  fire. 

(a)  Eyeball  (positive  visual  identification  is  required). 

(b)  Missiles  free  (positive  visual  identification  is  not 
required)  . 
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(3)  Enemy's  relative  bearing  from  the  subject. 

(a)  315° 

(b)  135° 

(4)  Enemy's  range  from  the  subject. 

(a)  Eyeball  or  within  4  nautical  miles. 

(b)  2  0  nautical  miles. 

(5)  Enemy's  true  heading  or  direction  of  flight. 

(a)  045° 

(b)  225° 

(6)  Number  of  enemy  aircraft . 

(a)  2 

(b)  6 

The  subjects  were  asked  for  their  rank,  total  number  of  flight 

hours  and  total  number  of  combat  missions.     The  following  background 

scenario  was  used  for  each  of  the  64  situations: 

"You  are  the  flight  leader  of  a  section     of  F-4's  armed  with  two 
Sparrows?-  and  two  Sidewinders  .3    Assume  for  this  exercise  that  the 
aircraft's  weapon  systems  are  up  in  every  respect.    You  are  providing 


A  section  normally  consists  of  two  F-4's 


2 

Beam  riding  guided  missile. 


3 

Heat  seeking  guided  missile 
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TARCAP  for  a  division  of  A-7's  who  have  just  completed  a  strike  and  are 
egressing  from  the  target  area.    You  are  feet-dry     over  North  Vietnam 
(20  nm  to  the  coast).     The  AA  and  SAM^  defenses  in  the  immediate  area 
are  light  to  moderate .    You  have  limited  GCI°  facilities  operating  for 
you  and  the  enemy  has  excellent  ground  radar  control. 

The  enemy  aircraft  are  assessed  to  be  MIG-21's  at  15,000  feet  and 
500  kts  .    You  are  10,000  feet  and  450  kts.  heading  for  your  carrier 
(360°  relative)  . 

The  weather  in  the  area  is  clear  and  15+  visibility.     There  are 
several  flights  of  attack  aircraft  still  feet-dry,  exact  position  unknown. 

The  MIG's  have  demonstrated  an  air-to-air  missile  capability." 
Figure  1  shows  the  way  the  tactical  situations  were  presented  to  the 
subjects,  analogous  to  a  radar  scope  or  maneuvering  board's  display. 

Two  major  evaluation  questions  were  asked  of  each  subject  in  con- 
junction with  each  of  the  64  situations  considered.    These  questions 
are  shown  in  Figure  2  . 

The  questionnaire  was  sent  to  a  number  of  Navy  fighter  squadrons 
on  both  the  East  and  West  Coasts.    The  aircraft  in  service  at  each  of 
the  squadrons  was  the  F-4  Phantom.     Thirty-six  pilots  responded.     These 
pilots  are  described  in  Appendix  B.     Table  1  contains  a  description  of 
the  pilots  by  rank,  hours  and  combat  missions.     The  overall  average 


4 
Over  land  vice  feet  wet  or  over  water, 


5 

Surface-to-Air  Missile. 


Ground  Control  Intercept 
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Figure  1 
TACTICAL  SITUATION  PRESENTATION 
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Figure  2 

QUESTIONS  ASKED  WITH  EACH  TACTICAL  DISPLAY 

A.         In  this  tactical  situation  (check  one  answer): 

I.         I'd  have  no  choice;  there  would  be  an  engagement. 

2  ..        Td  have  a  choice  on  whether  to  engage  or  not,  and  I'd 
engage. 

3..         I  wouldn't  engage. 

B -..        Indicate  what  aircraft  losses  you  would  predict,  if  there  were  an 
engagement . 

Enemy  Losses (no.  of  aircraft) 

Friendly  Losses (no.  of  aircraft) 
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re  spondee  had  a  rank  between  Lieutenant  and  Lieutenant  Commander 

7 
(3.47)    ,   1994.03  flight  hours  and  124.11  combat  missions.    Rank  was 

assumed  to  be  an  attribute  providing  ordinal  data. 

The  data  from  the  surveys  were  coded  and  placed  on  a  magnetic 
disk  for  statistical  analysis.    The  rules  of  engagement,  being  qualitative 
variables,  were  arbitrarily  coded  as  a  0  for  eyeball  and  1  for  missiles- 
free.    A  measure  of  central  tendency  (arithmetic  mean)  and  measure  of 
dispersion  (standard  deviation)  was  computed  for  each  biographical 
variable  (rank,  flight  hours,  combat  missions),  each  primary  tactical 
decision  and  each  secondary  enemy  and  friendly  kill  decision  made  for 
the  entire  sample.     Means  and  standard  deviations  of  these  variables 
and  decisions  were  then  computed  for  each  subset  of  the  sample  defined 
by  biographical  data  groupings,  i0e.  ,  all  Commanders,  all  Lieutenant 
Commanders,  all  Lieutenants,  all  Lieutenants   (junior  grade),  all  pilots 
with  more  than  4000  hours,  all  with  less  than  4000  and  more  than  3000, 
all  with  less  than  3  000  and  more  than  2  000,  all  with  less  than  2  000  and 
more  than  1000,  all  with  less  than  1000,  all  pilots  with  over  300  missions, 
all  with  less  than  300  and  more  than  200,  all  with  less  than  200  and 
more  than  100,  all  with  less  than  100  and  more  than  one  mission  and 
those  pilots  with  no  combat  missions.     Likewise,  the  mean  and  standard 
deviation  of  each  decision  was  computed  for  the  subset  of  the  sample 
defined  by  holding  each  of  the  situational  variables  constant  at  one  of 
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TABLE  I 


DESCRIPTION  OF  RESPONDING  FIGHTER  PILOTS 


Rank: 
Number: 
Hours:    Mean 

S.D 
Missions:    Mean 

S.D. 


0 


LT(jg) 

1 
450 

0 

0 

0 


LT 

21 

1394 

438 

96 

74 


29 
54 
52 
66 


LCDR 

10 
2675.50 
553.69 
163.40 

72.17 


CDR 

4 

3825 

466 

201 

106 


00 
61 
75 
81 


Hours:  0-999  1000-199! 

Number:                          5  17 

Rank7:    Mean               2.80  3.14 

S.D.                   .01  .88 

Missions:    Mean        0  127.07 

S.D.                 0  87.12 


2000-2999 
6 
3 


111 

54 


67 
47 
83 
78 


3000-3999 
6 

4 


242 
69 


33 
47 
00 
20 


0 


4000+ 

2 

5 

0 

106 

36 


50 
64 


Missions:  0  1-99  100-199  200-299  300+ 

Number:  6  7  13  8  6 

Rank7:    Mean  2.83  3.38  3.54  3.71  4.50 

S.D.  .37  .70  .63  .70  .50 

Hours:     Mean  763.33  2150.00  2065.77  2378.57  3250.00 

S.D.  177.87  971.43  870.87  832.28  50.20 


Ensign  =  1,  LT(jg)  =  2,  LT  =  3  ,  LCDR  =  4,  CDR  =  5 
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its  values  and  then  the  other  while  the  remaining  situational  variables 
were  free  to  assume  either  of  their  two  values  . 

In  attempting  to  find  a  relationship  between  the  biographical  and 
situational  variables  and  the  decisions  reached,  correlation  coefficients 
were  computed  between  these  variables  and  decisions.    These  correlation 
coefficients  were  computed  for  the  entire  sample,  for  each  subset  de- 
fined by  the  biographical  groupings,  and  for  each  respondee.    These 
findings  were  presented  to  a  panel  of  Naval  aviators  who  were  asked  for 
their  opinions  and  suggestions.     These  aviators  are  described  in  Appendix 
C.    From  their  suggestions  new  variables  made  up  of  combinations  of 
the  original  six  variables  were  added  to  the  data  base  and  the  same 
analysis  again  undertaken.     This  cycle  was  repeated  until  the  panel  con- 
sidered the  results  to  be  reasonable. 

In  an  attempt  to  distinguish  any  discernible  differences  between 
the  decisions  reached  for  each  biographical  subset  within  the  rank,  hour 
and  mission  groupings,  a  discriminant  analysis  was  performed.     In  this 
analysis,  a  linear  function  for  each  subset  within  a  group  is  computed 
using  the  subset  characteristic  (specific  rank,  hours  grouping,  or 
mission  grouping)  as  the  dependent  variable  and  the  decisions  reached 
as  the  independent  variables.     These  functions  are  then  evaluated  for 
each  case  and  the  hypothesis  that  the  subset  means  are  the  same  is 
tested  [9] . 


19 


In  an  attempt  to  predict  a  pilot  or  group  of  pilots'  decisions,  a- 
sequence  of  multiple  linear  equations  computed  using  stepwise  multiple 
regression  techniques  were  also  produced  for  the  entire  sample  and  each 
subset.    At  each  step  in  this  regression  procedure,  one  situation  variable 
is  added.    The  variable  added  is  the  one  which  makes  the  greatest  re- 
duction in  the  error  sum  of  squares  (the  variable  with  the  highest  partial 
correlation  with  the  dependent  variable  at  that  step) .     The  production 
stops  when  the  error  sum  of  squares  decreases  less  than  .001  or  until 
all  variables  are  exhausted  [10]  .    The  correlation  coefficient  between 
these  equation  predictions  and  the  actual  decision  reached  was  then 
computed  for  each  situation  examined.    These  equations  and  correlations 
were  then  presented  to  the  panel  of  aviators  for  their  evaluation. 
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m.     RESUL1SOT_DATA  ANALYSIS 

A.         VARIABLES  ADDED  AND  DELETED 

Since  bearing  and  heading,  although  quantitative  items  of  infor- 
mation  shown  to  he  instant  in  the  first  survey,  did  not  represent  con- 
tinuums that  could  be  eonveniently  statistiealiy  manipulated,  both 
were  deleted  in  favor  of  a  new  variable.    The  reason  this  was  done  is 
apparent  when  evaluating,  for  example,  bearings  of  001°  and  359°  . 
Mthough  separated  by  only  2°  on  a  oompass  rose,  a  guantity  evaluated 

i-    •!  i~     iv,o  ualupG  are  separated  by 
by  the  panel  o£  aviators  as  negligible,  the  value, 

357o     whieh  is  not  negligible.    Also,  the  two  variables'  effects  on  a 
pUofs  evaluation  appeared  to  be  interactive.    An  enemy  directly  behind 

\,,-,t„H  riifferently  if  the  enemy  was 
a  friendly  (bearing  180°)  was  evaluated  differently 

pointed  at  the  friendly  (heading  360°)  or  pointing  away  (heading  180°). 
Thls  pointing  information  is  not  directly  available  to  the  pilot  from  either 
his  cochpit  delays  or  GCI  input.    Yet  the  pilot  does  mentally  integrate 
the  bearing  and  heading  information  into  pointing  information.    Conse- 
quently, a  function  of  both  bearing  and  heading  was  created.    This 

i      "i,  Hpscribed  by  the  following  equation: 
function,  called  "danger  angle,     is  described  oy 

i_      a-    m  ■      (Qin  (bearing)/  I  sin  (bearing)  1   ) 
Danger  (bearing,  heading)  -    (sin  (bearing;/  { 

x    [bearing-  (heading  +  180)  modulus  360]. 

Similarly,  although  pilots  considered  range  and  fuel-remaining  as 
important  variables  independently,  the  aviator  panel  believed  that  a 


21 


link  existed  between  these  two  factors.    For  instance,  a  fighter  pilot's 
behavior  in  a  situation  of  long  range  with  low  fuel  remaining  would 
probably  be  different  from  that  under  a  situation  of  short  range  and  a 
large  amount  of  fuel  remaining.    Again,  this  would  be  a  case  of  pilot 
integration  of  a  cockpit  display  (fuel  remaining)  and  a  GCI  input  or 
radar  display  (range)  into  a  range  per  fuel  state  factor.     Consequently 
a  range-fuel  ratio  (MPG)  was  created  as  described  by  the  following 
equation: 

MPG  (range,  fuel  remaining)  =  range/fuel  remaining. 

As  shown  in  the  first  mail-out  questionnaire  results,  a  relationship 
between  the  enemy's  speed  and  heading  and  friendly  speed  and  heading, 
or  closure  rate  is  considered  by  fighter  pilots   (See  EXPERIMENTAL 
PROCEDURE  DETERMINING  DECISION  STRATEGY).    Again,   pilot  inte- 
gration of  cockpit  and  GCI  inputs  appears  to  occur:    the  pilot  projecting 
the  parallel  enemy  speed  vector  into  his  own.    As  a  result,  a  speed 
differential  function  was  created  for  the  tactical  scenario: 

Speed  (heading,  bearing)  =  500  x  cos  (heading)  -  (cos (bearing)/ 

|  cos  (bearing)  j  )  x  450. 

In  both  the  speed  and  danger  equations,  a  value  of  +1  was  assigned  to 

the  ratio  of  the  trigonometric  function  and  its  absolute  value  when  that 

function  was  equal  to  zero. 
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B.         CORRELATIONS 

The  primary  tactical  decision  made  by  the  subjects'  was  assumed 
to  be  ordinally  quantifiable  as  indicated  below: 

(1)  Pilot  has  no  choice,  an  engagement  would  occur. 

(2)  Pilot  has  a  choice  and  would  elect  to  engage  the  enemy. 

(3)  Pilot  would  not  engage  the  enemy. 

The  secondary  decision  was  in  two  parts,  number  of  enemy  losses  pre- 
dicted and  number  of  friendly  losses  predicted  if  an  engagement  were 
to  occur. 

The  biographical  data's  correlation  with  the  decision  is  shown  in 
Table  II  for  all  pilots. 

TABLE  II 

CORRELATION  OF  BIOGRAPHICAL  DATA  WITH  PRIMARY 
AND  SECONDARY  DECISIONS 


Tactical 

Enemy 

Friendly 

Decision 

Kills 

Kills 

-.0392 

.0541 

.2195 

-.0394 

.0628 

.  1685 

-.0333 

.0801 

.1265 

Rank 
Hour 
Missions 


The  lack  of  correlations  significantly  different  from  zero  is  striking. 
Only  with  the  relationship  of  rank  to  friendly  kills  predicted  can  a 
coefficient  of  correlation  be  found  of  any  sizeable  magnitude  (+.2195). 
Discriminant  function  analysis  revealed  no  significant  differences 
between  the  pilots  when  they  were  categorized  by  rank,  hour  or  mission 
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with  respect  to  the  decisions  they  made  (at  a  significance  level  of 
<  .05)   (See  Appendix  D)  .    The  panel  of  aviators  considered  this  (post 
hoc)  to  be  a  result  of  standardized  training  and  operational  squadron 
procedures.     They  also  expected  a  positive  correlation  between  rank 
and  expected  friendly  kills  as  a  result  of  more  realistic  evaluation 
techniques  acquired  with  time  spent  in  operational  billets.    A  decline 
in  youthful  optimism  as  one  ascends  the  rank  and  age  structure  was  also 
expected  to  enhance  this  postulated  positive  correlation.    Appendix  F 
contains  the  means  and  standard  deviation  of  the  decisions  reached  by 
the  biographical  groupings. 

The  situational  variables'  correlations  with  the  decisions  are 
shown  in  Table  III. 

TABLE  III 

CORRELATIONS  OF  SITUATIONAL  VARIABLES  WITH 

PRIMARY  TACTICAL  DECISIONS  AND  SECONDARY 

EXPECTED  ENEMY  AND  FRIENDLY  KILL  DECISIONS 


Tactical 

Enemy 

Friendly 

Decision 

Kills 

Kills 

MPG 

.4259 

-.2087 

-.1237 

Speed 

.2559 

-.2339 

.0553 

Danger 

.2864 

-.2857 

-.0201 

Fuel  Rei 

maining 

-.1708 

.3400 

-.0203 

Range 

.4209 

-.0163 

-.1746 

Number 

of  Enemy 

.1223 

.0231 

.2447 

Rules  of  Engagement 

-.0760 

.2287 

-.0512 
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The  panel  of  aviators,  however,  felt  a  chronological  relationship 
existed  between  the  decisions,  that  is,  the  pilot  evaluated  the  situation 
as  a  whole  and  then  reached  his  primary  tactical  decision.    After  that 
decision  had  been  reached,  and  with  that  result  kept  in  mind,  it  was 
thought  that  the  pilot  then  made  his  secondary  decisions.     Suggestions 
by  the  author  that  the  decisions  of  kill  possibilities  would  precede  and 
influence  the  tactical  decision  were  strongly  rejected  by  the  panel  as 
unrealistic  from  their  own  experiences.     Table  IV  shows  the  correlation 
between  the  primary  and  secondary  decisions. 

TABLE  IV 

CORRELATION  BETWEEN  PRIMARY  AND 
SECONDARY  DECISIONS 

Enemy  Kills  Friendly  Kills 

Tactical  Decision  -.4247  -.2307 

The  reader  should  note  that  the  biographical  variable  of  rank  is 
the  third  most  important  variable  when  predicting  secondary  friendly 
kills. 

C.         VARIABLE  MANIPULATION 

The  effects  of  holding  specific  situation  variables  constant  on  the 
decisions  made  is  shown  in  Table  V.     MPG  was  not  held  constant  since 
that  would  imply  a  simultaneous  manipulation  of  range  and  the  fuel- 
remaining  variable. 
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The  most  striking  phenomena  evident  in  Table  V  is  the  ratio 

between  predicted  enemy  kills  and  predicted  friendly  kills  of  over  four 

to  one.    As  explained  by  the  senior  member  of  the  panel  of  aviators 

after  observing  this  phenomena: 

"A  pilot  would  be  a  fool  to  go  into  combat  is  he  didn't  think 
the  odds  were  stacked  heavily  in  his  favor." 

The  inverse  relationship  evident  in  this  table  between  range  and 
friendly  kills  was  considered  by  the  aviators  to  be  a  function  of  aircraft 
type..    The  F-4  was  considered  by  them  to  be  more  of  a  weapons  platform 
than  a  classical  dog-fighter  while  the  MIG-21,  a  smaller  and  more 
maneuverable  airplane,  is  better  suited  for  close-in  fighting.    The 
shorter  range  and,   consequently,  time  available  to  the  F-4  weapons 
systems  decreases  some  of  this  system's  advantages  over  a  MIG-21 
resulting  in  a  greater  enemy  opportunity  to  attack  the  friendly  forces. 
Interesting  conjectures  can  be  made  about  the  results  that  might  be 
observed  if  the  friendly  plane  were  an  F-8,  a  true  dog-fighter.  .  .The 
slight  increase  in  enemy  kills  with  decreasing  range  which  is  evident 
in  Table  V  was  felt  by  the  panel  of  aviators  to  be  the  result  of  an  in- 
creased friendly  missile  kill  probability. 

The  pushing  of  the  tactical  decision  to  the  "must  engage"  pole 
with  shorter  ranges  was  felt  to  be  an  indication  that  a  pilot  naturally 
considers  danger  more  imminent  as  the  enemy  closes;  the  shorter  ranges 
decreasing  the  protector's  options. 
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TABLE  V 


VARIABLE  MANIPULATION -AVERAGE 
VALUE  PER  SITUATION 


Variable  Value         Tactical  Decision         Enemy  Kills         Friendly  Kills 

Range 


4 
20 


1.66 
2.32 


Rules  of  Missile  Free     1.93 

Engagement       Eyeball  2.05 


Number  of 
Enemy 

2 
6 

1.89 
2.09 

Speed 

0 
0 

1.71 
2.28 

Danger 

0 
0 

2.03 
1.83 

Fuel 
Remaining 

1000 
2500 

2.12 
1.86 

All  Free 

Mean 
S.D. 

1.99 
.79 

1.09 

.37 

1.05 

.15 

1.31 

.23 

.84 

.29 

1.05 

.11 

1.10 

.41 

1.42 

.24 

.73 

.28 

.94 

.23 

1.19 

.29 

.73 

.27 

1.42 

.25 

1.07 

.26 

1.01 

.62 
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The  differences  displayed  in  the  decisions  as  rules  of  engagement 
are  varied  were  also  felt  by  the  panel  to  reflect  aircraft  type.    The  F-4 
is  at  its  best  when  able  to  deliver  the  long  range,  unsuspected  missile 
under  missiles  free  conditions.    A  40%  drop  in  predicted  enemy  kills  can 
be  seen  in  Table  V  as  the  rule  of  engagement  assumes  the  eyeball  con- 
dition.   Waiting  for  ranges  to  shorten  for  eyeball  identification  means 
losing  the  advantage  of  the  long  range  punch.    The  decreased  range 
forced  on  the  pilot  shows  in  Table  V,  as  before;  in  an  increase  in  pre- 
dicted friendly  kills  but  not,  surprisingly,  in  a  tactical  evaluation  pushed 
towards  the  "must  engage"  pole. 

The  greatly  increased  number  of  predicted  friendly  kills  when  the 
number  of  enemy  is  high,  as  shown  in  Table  V,  is  considered  to  be  a 
direct  reflection  of  superiority  in  aircraft  numbers.     No  matter  how 
effective  a  protector  may  be,  once  the  protectors  are  saturated,   some 
of  the  enemy  will  reach  the  vulnerable  friendly  forces.     Also,  the  more 
enemy  present  the  more  enemy  the  friendlies  may  destroy.     The  tactical 
decision's  migration  towards  the  "not  engage"  pole  was  felt  by  the  panel 
to  be  a  sign  of  caution  by  the  friendly  in  light  of  a  large  enemy  force; 
the  gains  of  engagement  being  more  than  offset  by  the  possible  friendly 
losses  . 

The  speed  differences  between  the  friendly  and  enemy  forces 

appear  to  push  the  primary  tactical  decision  towards  the  two  poles  of 

engage  or  not  engage.    A  large  speed  differential  coupled  with  opening 
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ranges  implies,  according  to  the  panel,  a  friendly  decision  of  leaving 
the  friendly  forces  exposed  for  a  long  chase  or  staying  with  the  friendlies 
and  not  engaging  the  enemy.    The  high  closure  rate,  however,  would 
force  the  friendly  to  engage  but,  as  shown  in  the  extremely  high  predicted 
enemy  kills  in  Table  V,  would  place  the  enemy  in  the  F-4's  most  optimal 
firing  envelope . 

The  enemy  pointing  at  the  friendly  forces  places  the  friendlies  in 
the  enemy's  best  firing  envelope  and,  as  shown  in  Table  V,  seem  to 
force  the  friendly  tactical  decision  to  the  defensive  "must  engage"  pole. 
As  a  result,  a  rise  in  both  predicted  enemy  and  friendly  kills  can  be 
seen.    The  enemy  pointing  away  from  the  friendlies  with  its  implied 
opening  ranges  drives  the  tactical  decision  to  the  "not  engage"  side 
with  concurrent  drops  in  both  predicted  kills . 

The  low  enemy  kills  and  "not  engage"  decision  evident  with  low 
fuel  state  was  interpreted  by  the  panel  as  a  sign  for  caution  for  the 
friendly  pilot.    Friendly  kills  rise  in  this  low  fuel  state  not  only  from 
enemy  attacks  but  from  destruction  due  to  fuel  exhaustion  in  combat. 
Higher  fuel  states  give  the  pilot  a  greater  margin  in  combat  for  increased 
maneuvering  and  chasing,  resulting  in  the  increased  predicted  enemy 
kills  and  greater  friendly  protection. 
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D.         MULTIPLE  REGRESSION 

From  the  multiple  regression  analysis  a  ranking  of  situational 
variable  importance  was  made  for  all  pilots  and  for  each  rank,  hour  and 
mission  group.     Table  VI  gives  these  rankings  for  all  pilots  and  for  the 
rank  groupings.    Appendix  G  contains  the  hour  and  mission  rankings.    As 
can  be  seen,   MPG  or  range,  one  a  function  of  the  other,  play  an  important 
role  in  the  tactical  decision  evaluation.     Danger,  except  in  the  case  of 
Commander  where  it  switches  position  with  speed,  is  next  most  important 
followed  by  speed. 

In  the  importance  of  variables  for  secondary  decision  evaluation, 
the  tactical  decision  is  the  most  important  in  evaluating  enemy  kills 
while  the  number  of  enemy  is  the  only  variable  more  important  in  evalu- 
ating friendly  kills.    This  may  reflect  the  pilot's  integration  of  his 
primary  decision  in  his  next  evaluation,  i.e.  ,  the  pilot's  "mental  set" 
before  actually  making  his  next  evaluations  . 

From  this  multiple  regression  analysis  predictor  equations  were 
constructed.     Described  in  Appendix  E,  the  equations  were  used  in 
predicting  the  decisions  made  on  the  decision  survey.    The  obtained 
multiple  correlations  are  described  in  Table  VII  and  Appendix  H. 
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TABLE  VI 

VARIABLE  RANKINGS  FOR  ALL  PILOTS  AND 
RANK  GROUPS  IN  DECISIONS  REACHED 

A.         Tactical  Decision 

ALL  CDR  LCDR  LT  LT(jg) 


MPG 

1 

7 

1 

7 

Danger 

2 

3 

2 

2 

2 

Speed 

3 

2 

3 

3 

6 

Range 

4 

1 

1 

5 

1 

Number  of  Enemy 

5 

4 

4 

4 

4 

Rules  of  Engagement 

6 

6 

6 

6 

5 

Fuel  Remaining 

7 

5 

5 

7 

3 

B .         Enemy  Kills 

ALL 

CDR 

LCDR 

LT 

LT(j 

Tactical  Decision 

1 

1 

1 

1 

1 

Fuel  Remaining 

2 

4 

3 

2 

2 

Rules  cf  Engagement 

3 

6 

4 

3 

8 

Danger 

4 

5 

2 

4 

4 

Speed 

5 

8 

5 

5 

3 

Range 

6 

2 

6 

6 

6 

Number  of  Enemy 

7 

3 

7 

7 

5 

MPG 

8 

7 

8 

8 

7 

C.        Friendly  Kills 


ALL  CDR  LCDR  LT  LT(jg) 


Number  of  Enemy 

1 

1 

2 

1 

1 

Tactical  Decision 

2 

2 

1 

2 

4 

Speed 

3 

4 

4 

3 

3 

Rules  of  Engagement 

4 

3 

7 

4 

5 

Fuel  Remaining 

5 

8 

5 

5 

8 

Danger 

6 

5 

8 

6 

6 

Range 

7 

7 

3 

7 

2 

MPG 

8 

6 

6 

8 

7 
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TABLE  VII 

CORRELATION  OF  PREDICTED  VALUES 

FOR  DECISIONS  AND  ACTUAL  DECISIONS 

MADE  BY  ALL  PILOTS  AND  RANK  GROUPINGS 

ALL  CDR  LCDR             LT  LT(jg) 

Tactical  Decision               .6032  .5794  .6236  .6101  .7568 

Enemy  Killed                         .6027  .5859  .5654  .6527  .8193 

Friendly  Killed                      .4015  .4391  .5299  .3732  .5284 

It  appears  that  the  equations  are  fairly  successful  for  the  groups  studied 

in  predicting  their  tactical  and  enemy  kill  decisions. 

In  an  attempt  to  secure  a  standard  of  comparison  for  these  equations, 

the  panel  of  aviators  was  used  to  predict  one  of  the  biographical  group's 

decisions.    The  members  of  the  panel,  who  were  associated  with  the 

data  at  some  length,  considered  themselves  to  be  closest  to  the 

Lieutenant  Commander  group  in  rank,  hours,  and  missions.    All  of  the 

Lieutenants  on  the  panel  were  in  the  current  Lieutenant  Commander 

selection  zone.    They  attempted  to  predict  the  Lieutenant  Commander 

group's  decision  rather  than  only  making  their  own  evaluations  of  the 

situations.    The  following  correlations  between  the  panel's  predictions 

and  the  actual  decisions  made  by  the  Lieutenant  Commander  group  were 

found: 

Tactical  Decision  .5863 

Enemy  Killed  .52  00 

Friendly  Killed  .4327 
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En.  essence,  the  predictor  equations  fared  better  than  the  predicting 
panel  of  aviators  (See  Table  VII) . 
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IV..    EXPERIMENTAL  PROCEDURE-ARTIFICIAL  INTELLIGENCE  APPLICATION 

One  writer  [11]  has  described  and  defined  artificial  intelligence  in 

the  following  way: 

"A  human  being  can  think,  learn,  and  create  because  the  program 
his  biological  endowment  gives  him,  together  with  the  changes  in 
that  program  produced  by  interaction  with  his  environment  after 
birth,  enables  him  to  think,  learn,  and  create.     If  a  program  thinks, 
learns,  and  creates,  it  will  be  by  virtue  of  a  program  that  endows 
it  with  these  capacities.    Clearly  this  will  not  be  a  program  -  any 
more  than  the  human's  is  -  that  calls  for  highly  stereotyped  and 
repetitive  behavior  independent  of  the  stimuli  coming  from  the 
environment  and  the  task  to  be  completed.     It  will  be  a  program 
that  makes  the  system's  behavior  highly  conditional  on  the  task 
environment  -  on  the  task  goals  and  on  the  clues  extracted  from 
the  environment  that  indicate  whether  progress  is  being  made  towards 
those  goals.     It  will  be  a  program  that  analyzes,  by  some  means, 
its  own  performance,  diagnoses  its  failures,  and  makes  changes 
thai:  enhance  its  future  effectiveness." 

With  the  goal  of  the  last  sentence  above  in  mind,  the  panel  of 

aviators,  all  of  whom  have  had  student  aviator  instructor  billets, 

described  a  student  pilot's  behavior  in  interaction  with  his  instructor. 

Once  past  the  basic  ground  schools,  flight  training  appeared  to  entail 

corrective  instruction  of  the  student's  cockpit  actions.    The  corrective 

instruction  can  be  broken  down  into  three  general  areas: 

(1)  The  student  did  the  opposite  of  what  he  should  have  done. 

(2)  The  student  did  too  much  of  what  he  should  have  done. 

(3)  The  student  did  not  do  enough  of  what  he  should  have  done. 
For  example;   (1)  pushing  the  stick  to  the  left  vice  the  right,    (2)  applying 
too  much  throttle,  and  (3)  not  using  enough  flaps.     Such  mistakes  as 
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forgetting  to  lower  the  landing  gear  were  considered  as  combinations  or 
extremes  of  the  three  . 

The  student's  response  to  the  three  corrections  were: 

(1)  Reversing  the  sign  or  direction  of  what  was  incorrectly  done. 

(2)  Lessening  the  magnitude  of  what  was  incorrectly  done. 

(3)  Increasing  the  magnitude  of  what  was  incorrectly  done. 

Also,  as  the  student  pilot  gains  more  experience  his  confidence  level  in 

himself  and  aircraft  rises.    His  corrections  grow  smaller  in  magnitude 

resulting  in  a  smoother  development.    As  stated  by  one  of  the  aviators 

on  the  panel: 

"Any  movement  which  is  not  smooth  and  gentle  in  an  aircraft  is 
probably  an  incorrect  movement." 

In  advanced  Naval  pilot  training  where  such  topics  as  tactics  and 
gunnery  are  taught,  the  three  instructive  areas  are  thought  to  still  exist. 
However,  at  this  level  the  pilot  is  developing  his  evaluative  skills 
rather  than  his  motor  skills.     Since  multi-level  decisions  are  involved, 
the  student  is  not  allowed  to  proceed  to  the  next  level  until  the  present 
level  decision  or  action  is  correct.    Figure  3  displays  this  macro-level 
education  concept. 

This  concept,  if  captured  in  a  program,  would  enable  the  program 
to  analyze  its  performance  and  diagnose  its  failures   (instructor  correction) 
and  make  changes  to  enhance  its  future  effectiveness  (student  modifi- 
cation) .    This  program  could  show  some  attributes  (confidence  and 
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Figure  3 

MACRO-LEVEL  PILOT  EDUCATION 


A: 


Present  situation  to  student. 


B:  Student  evaluates  situation  and  makes  primary  decision 


C:         Is  decision  correct? 


D:         Student  gains  confidence  and  experience 


E:  Student  makes  secondary  decisions. 


G:         Student  modifies  own  evaluation  techniques 
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experience,  learning,  fallibility)  that,  if  witnessed  in  a  living  organism, 
would  certainly  be  termed  intelligent.    Hence,  an  artificial  intelligence 
approach  to  this  macro-level  pilot  learning  concept  was  initiated. 

Certain  restrictions  were  placed  on  this  artificial  intelligence 
programming  attempt.     Since  possible  graphics  adaptations  were  en- 
visaged, memory  size  limitations  were  desired.    The  available  graphics 
terminal  at  Naval  Postgraduate  School  is  an  AGT-10  with  limited  memory. 
Also,  as  brought  out  by  the  panel  of  aviators,  a  pilot  does  not  consciously 
search  through  his  memory  looking  for  a  situation  or  solution  but  rather 
appears  to  act  at  a  level  of  perceptual  decision  making  [7]  .     Likewise, 
the  panel  considered  the  classical  tree  searching  and  minimaxing  pro- 
cedures as  unrealistic  for  this  problem.     Computer  run  time  was  to  be 
as  short  as  possible  for  both  the  practical  consideration  of  program  turn 
around  time  and  because  of  the  desire  for  instantaneous  decisions  as  in 
high  speed  air-to-air  combat.     Consequently,  a  polynomial  evaluation 
with  adaptation  of  the  coefficients  technique  was  chosen  along  the  lines 
of  Samuel's  checker  player  [2]  .    The  coefficient  would,  in  essence,  act 
as  a  quasi-memory ,  reflecting  change  but  not,  necessarily,  history. 
FORTRAN  IV  was  chosen  for  the  program  language  over  ALGOL  and  LISP 
because  of  future  graphics  applications  where  no  ALGOL  or  LISP  compilers 
would  be  available. 
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The  "instructor"  would  be  one  of  the  pilot  groups  for  which  a 
predictive  equation  existed .    The  instructor's  memory  would  contain 
the  predictive  equation  and  all  of  the  decisions  made  concerning  the 
64  situations  presented  in  the  second  survey.    The  "student"  would 
contain  a  predictive  equation,  either  from  actual  data  analysis,  or  a 
dummy  one.     It  would  have  no  decision  history  memory. 

A.         PROGRAM  DESCRIPTION 

The  following  matrices  and  vectors  of  interest  are  used  throughout 
the  program: 

(1)  sit  (i,j)         i  =  1,   64;  j  -  1,   6 

This  matrix  contains  the  six  situational  variables  of  the 
second  array: 

3  =  1-  fuel  remaining, 

j  -  2  -  range, 

j  -  3  -  bearing, 

j  =  4  -  heading , 

j  =  5  -  number  of  enemy, 

j  =  6  -  rules  of  engagement. 

They  are  presented  in  the  64  different  situations,  i  going 
from  1  to  64  . 

(2)  mem  (i,j)         i  =  1,   64;  j  =  1,   3 

This  matrix  contains  the  three  decisions  made  by  the 
instructor  over  the  64  situations: 

j  =  l-  primary  tactical  decision, 

j  =  2-  secondary  enemy  kills  decision, 

j  =  3  -  secondary  friendly  kills  decision. 
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(3)  prico  (1)         i  =  1,   8 

This  vector  contains  the  instructor's  primary  tactical 
decision  predictive  equation  coefficients  for  the  variables 
below: 

i  =  1  -  a  constant 

L  =  21  -  fuel  remaining 

1=3  -  range 

1  =  4  -  number  of  enemy 

i  =  5  -  rules  of  engagement 

i  =  6  -  danger 

i  =  7  -  speed 

1=8-  MPG 

(4)  startp  (i)         i  =  1,   8 

Completely  analogous  to  prico ,  this  vector  contains  the 
student's  primary  tactical  decision  coefficients. 

(5)  secoen  (i)         i  =  1 ,   9 

This  vector  contains  the  instructor's  secondary  enemy  kills 
decision  predictive  equation  coefficients  for  the  same 
variables  as  in  prico.    When  i  =  9,  the  coefficient  is  for  the 
primary  tactical  decision  reached. 

(6)  secofr  (i)         i  =  1,   9 

This  vector  is  the  same  as  secoen  but  for  the  secondary 
friendly  kills  decision  predictive  equation. 

(7)  starse  (i)         i  =  1,   9 

Completely  analogous  to  secoen,  this  vector  contains  the 
student's  secondary  enemy  kills  decision  coefficients. 

(8)  starsf  (i)         i  =  1,   9 

Completely  analogous  to  secofr,  this  vector  contains  the 
student's  secondary  friendly  kills  decision  coefficients. 

(9)  kor  (i)         i  =  1 ,  8 

This  vector  is  the  vehicle  for  the  instructor's  correction  of 
the  student's  startp  coefficients. 

(10)        kore  (i)         i  =  1,9 

This  vector  is  the  vehicle  for  the  instructor's  correction  of 
the  student's  starse  coefficients. 
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(11)        korf  (i)         i  =  1,9 

This  vector  is  the  vehicle  for  the  instructor's  correction  of 
the  student's  starsf  coefficients. 

1 .         Main  Program 

The  main  program  contains,  in  essence,  the  previously 
described  macro-level  pilot  education  concept.    The  64  situations  are 
read  in  (sit (i , j))  followed  by  the  instructor's  decisions  over  these  sit- 
uations (mem(i,j))  followed  by  the  instructor's  predictive  equations 
(prico(i),   secoen(i),  secofr(i)).    This  constitutes  the  instructor's  memory 
and  expertise,  his  references  and  standards.     Next,  the  student  is 
brought  into  the  scene  by  the  reading  of  his  starting  evaluation  equations 
(startp(i),   starse(i),   starsf  (i)) ,  these  data  representing  the  starting 
student  at  the  beginning  of  the  interactive  instructor-student-situational 
process . 

Next,  for  each  of  900  trials,  one  of  the  64  situations  is  picked 

at  random  and  presented  to  the  student.    After  calculating  danger,   speed 

and  MPG;  a  skill  hopefully  developed  through  previous  training,  but 

probably  never  applied,  the  student  makes  the  primary  tactical  decision 

for  situation  I: 

IDECIS  =  startp  (1)  +  .5  +  startp  (2)  x  sit  (I,  1) 

+  startp  (3)  x  sit(I,2)  +  startp  (4)  x  sit(I,5) 

+  startp  (5)  x  sit  (I,  6)  +  startp  (6)  x  danger 

+  startp  (7)  x  speed  +  startp  (8)  x  MPG. 

A  subroutine  INSTRU  is  now  called.    This  is  equivalent  to  the 

instructor  grading  the  student  by  saying  "You've  acted  in  the  wrong 
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direction";  "too  much";  "too  little";  or  "correct."    From  this  grade  the 
student  is  either  sent  on  to  make  his  secondary  decisions  (correct)  or 
his  primary  evaluation  equation  is  sent  to  KORECI  subroutine  for  modi- 
fication.    If  the  student's  grade  is  "not  correct/'  upon  returning  from 
modification  he  returns  to  the  beginning  to  have  another  situation 
presented  without  attempting  to  make  a  secondary  decision.     If  no  modi- 
fication is  needed,  the  student's  experience  and  confidence  level  is 
increased  and  the  secondary  decisions  are  made  for  situation  I: 

IIEK  =  starse  (1)  +  .5  +  starse  (2)  x  sit(I,l) 

+  starse  (3)  x  sit  (1 ,2)  +  starse  (4)  x  sit  (1 ,5) 

+  starse  (5)  x  sit  (I,  6)  +  starse  (6)  x  danger 

+  starse  (7)  x  speed  +  starse  (8)  x  MPG 

+  starse  (9)  x  IDECIS  (enemy  kills), 

IIFK  =  starsf  (1)  +  .5  +  starsf  (2)  x  sit (1,1) 

+  starsf  (3)  x  sit  (1 ,2)  +  starsf  (4)  x  sit  (I,  5) 
+  starsf  (5)  x  sit  (1 , 6)  +  starsf  (6)  x  danger 
+  starsf  (7)  x  speed  +  starsf  (8)  x  MPG 
+  starsf  (9)  x  IDECIS  (friendly  kills)  . 

As  in  INSTRU,  these  decisions  are  graded  by  subroutine  GRADER. 

These  grades  are  considered  after  which  either  the  student's  equations 

are  modified  by  KOREII  or  KORFII  or  his  experience  and  confidence  levels 

are  increased.     In  either  case,  the  student  is  sent  back  to  the  beginning 

for  a  new  situation.     The  grading  and  correcting  of  the  secondary  enemy 

kill  and  friendly  kill  decisions  are  handled  independently,  hence 

correction  may  take  place  in  one  but  experience  and  confidence  may  rise 

in  the  other. 
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2  ..         Subroutine  INSTRU 

INSTRU  first  determines  if  it  agrees  with  the  student'  s 
primary  tactical  decision  by  checking  the  instructor's  memory  for  the 
same  situation  (mem(I,l))  .     If  the  instructor  and  the  student  agree  this 
fact  is  immediately  returned  to  the  main  program.     If  there  is  no  agree- 
ment, the  instructor  undertakes  a  coefficient  by  coefficient  comparison 
of  its  own  predictive  equations   (prico)  with  the  student's  evaluation 
equation  (startp)  for  the  primary  decision.     First  compared  as  to  sign 
and  then  as  to  magnitude,  a  grade  is  sent  back  to  the  main  program  for 
each  coefficient;  e.g.,  wrong  sign,  too  small,  too  large,  or,  agrees 
with  instructor's  coefficient.    The  coefficient  comparison  is  undertaken 
only  when  the  student's  primary  decision  does  not  agree  with  the 
instructor's  . 

3.  Subroutine  GRADER 

Subroutine  GRADER  is  essentially  the  same  as  subroutine 
INSTRU  but  deals  with  the  grading  and  coefficient  comparison  of  the  two 
secondary  decision  equations.    The  two  secondary  decisions  are  handled 
independently. 

4.  Subroutine  KORECI 

Subroutine  KORECI  handles  the  modification  of  the  student's 
equation  for  his  primary  tactical  decision.     A  student's  attempted  cor- 
rection of  the  same  error  is  not  necessarily  stable  in  magnitude, 
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especially  when  he  is  first  beginning  to  learn  to  fly.    As  a  student's 
experience  and  confidence  grows,  however,  the  modifications  become 
more  consistent  and  decreasing  in  magnitude.     Consequently,  an 
experience  modification  factor,  ZZ,  is  set.     Previous  runs  had  shown 
that  multiplicative  modifiers  differing  more  than  about  .11  from  1  caused 
an  erratic  and  sudden  convergence  of  the  student's  coefficient  to  that 
of  the  instructor's.     Therefore,  it  was  decided  to  limit  the  modification 
factor  to  the  closed  interval  [9  ,   1.1]. 

The  experience  modification  factor  is  set  by  taking  a  uniformly 
distributed  random  number  from  the  open  interval  (0,1).    A  different 
number  is  chosen  each  time  KORECI  is  called.     This  number  is  then 
divided  by  the  sum  10  +  .  1  x  experience  and  confidence  level.    This 
experience  and  confidence  level  (X)  is  simply  the  number  of  correct 
primary  tactical  decisions  that  have  been  made.     Hence  the  experience 
modification  factor  (ZZ)  is  in  the  open  interval  (0 ,  .1).    Adding  or  sub- 
tracting ZZ  from  one  produces  a  modification  factor  in  the  open  interval 
(.9,1.1)  that  approaches    unity.       For  example;  a  starting  random  number 
of  .5  with  no  experience  and  confidence  gives  a  modification  factor  of 
.95  or  1.05,  a  starting  random  number  of  .5  with  an  experience  and 
confidence  level  of  200  gives  a  modification  factor  of  .98  or  1.02,  and 
a  starting  random  number  of  .9  with  an  experience  and  confidence  level 
of  1000  gives  a  modification  factor  of  .99  or  1.01. 
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After  the  experience  modification  factor  is  set,  the  student's 
coefficients  are  checked  to  see  if  the  variable  paired  with  that  co- 
efficient is  even  being  considered,  i.e.,  is  the  coefficient  zero?    If 
so,   subroutine  NOVICE  is  called,  which  means  that  more  intensive 
instruction  is  given,  or,  a  return  is  made  to  a  more  elementary  student 
state.     If  not,  the  coefficients'  gradings  are  examined  and  the  co- 
efficients are  modified  according  to  the  results  of  these  gradings.     If 
a  wrong  sign  is  indicated,  the  student's  coefficient  is  multiplied  by  -1. 
If  the  coefficient  is  too  great  in  magnitude,  the  student's  coefficient 
is  multiplied  by  the  modification  factor  1-ZZ.     If  the  coefficient  is  too 
small,  the  modification  factor  is  1+ZZ .    A  coefficient  determined  by 
INSTRU  as  being  correct  is  not  modified. 

5.  Subroutine  KOREII 

Subroutine  KOREII  is  essentially  the  same  as  subroutine 
KORECI  but  handles  the  secondary  enemy  kill  decisions.    The  experience 
and  confidence  level  is  the  total  number  of  correct  secondary  enemy  kill 
decisions  made  by  the  student. 

6.  Subroutine  KORFII 

Subroutine  KORFII  is  identical  to  subroutine  KOREII  but 
handles  the  secondary  friendly  kill  decisions.     The  experience  and  con- 
fidence level  is  the  total  number  of  correct  secondary  friendly  kill 
decisions  made  by  the  student. 
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7.  Subroutine  NOVICE 

The  object  of  this  subroutine  is  to  instruct  the  student  to 
consider  a  variable  that,  up  to  this  time,  was  not  being  considered 
(coefficient  of  zero).     Since  data  had  revealed  that  out  of  330  coefficients 
considered  in  the  analysis  of  all  pilots  and  rank,  hours,  and  mission 
groups,  329  fell  into  the  interval  (-30,30),  a  uniformly  distributed 
random  number  is  selected  from  the  interval  (-30,3  0)  each  time  NOVICE 
is  called.     This  is  then  given  the  sign  of  the  instructor's  coefficient 
for  that  variable  and  assigned  to  the  student's  coefficient  in  lieu  of  the 
previous  zero  value. 

8.  Subroutine  RANDOM 

This  subroutine,  a  slightly  modified  version  of  IBM's  RANDU 
[12],  simply  generates  a  uniformly  distributed  random  number  in  the 
interval  (0,1). 

(All  routines  can  be  found  in  the  Computer  Printout  Section 
of  this  thesis)  . 

B.         RUNNING  THE  PROGRAM 

The  program  was  run  with  the  average  pilot  for  the  entire  sample, 
the  statistically  average  pilot  from  each  rank  group,  the  statistically 
average  pilot  from  each  hour  group  and  the  statistically  average  pilot 
from  each  mission  group  as  instructors  with  the  same  groups'  predictor 
equations  as  starting  coefficients  for  the  students.    Therefore,  each 
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instructor  was  run  with  all  of  these  average  pilots  as  students  .    As  can 
be  seen  in  Figures  4  through  15  and  Table  VIII,  little  differences  exist 
between  the  instructor-student  performances. 

As  a  result  of  these  runs,  a  synthetic  beginning  student  was  con- 
structed with  all  starting  coefficients  set  to  zero.    Using  this  beginner 
model  as  a  student,  the  program  was  rerun  with:     1)  the  average  pilot 
for  the  entire  sample,   2)  the  Commander  group,  3)  the  4  000-plus  hour 
group,  and  4)  the  300-plus  mission  group  as  instructors.     The  latter 
three  groups  were  chosen  for  their  expertise  under  the  assumption  that 
a  pilot  who  has  attained  higher  rank,  flown  4000  non-fatal  hours  or 
survived  300  missions  must  possess  some  expertise  in  flight.    The 
results  obtained  from  this  "beginner"  student  run  were  plotted  and 
compared  to  those  of  the  average  student  (See  Figures  4-15,   16-27  and 
Table  VIII)  .    These  comparisons  and  plots  were  then  presented  to  the 
panel  of  aviators  for  their  comments  and  criticisms. 
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TABLE  "VIII 

RESULTS  OF  PROGRAM  RUNS  WITH  TWO  STUDENTS 
AND  FOUR  INSTRUCTORS 

Average  Pilot  as  Student: 

Time*  to  Make  First  Correct  Decision 

Instructor:                            Average  Pilot  CDR  4000+  300  + 

Primary  Decision                         5  5  5  5 

Secondary  Enemy  Kills               5  5  5  5 

Friendly  Kills           5  5  5  5 

Time*  to  Attain  90%  of  Final  Correct  Percentage  After  First  Correct 

Decision 

Instructor:  Average  Pilot  CDR  4000+  300  + 

Primary  Decision  5  62  40  96 

Secondary  Enemy  Kills  19  39  8  73 

Friendly  Kills  5  57  8  95 

Final  Correct  Decision  Proportions 

Instructor:  Average  Pilot  CDR  4  000+  300+ 

Primary  Decision  .85  .67  .87  .83 

Secondary  Enemy  Kills  .72  .63  .60  .55 

Friendly  Kills  .94  .71  .62  .64 

Synthetic  Beginner  as  Student: 

Time*  to  Make  First  Correct  Decision 

Instructor:  Average  Pilot 

Primary  Decision  2  00 

Secondary  Enemy  Kills  12  9 

Friendly  Kills       157 

Time*  to  Attain  90%  of  Final  Correct  Percentage  After  First  Correct 

Decision 

Instructor:  Average  Pilot 

Primary  Decision  40 

Secondary  Enemy  Kills  13  7 

Friendly  Kills         56 

Final  Correct  Decision  Proportions 

Instructor:  Average  Pilot 

Primary  Decision  .84 

Secondary  Enemy  Kills  .  72 

Friendly  Kills  .93 

*Time  is  in  units  of  trials  for  the  primary  decision  and  in  units  of  correct 
primary  decisions  for  the  secondary  decisions. 
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CDR 

4000+ 

300  + 

221 

199 

206 

194 

172 

149 

177 

161 

177 

CDR 

4000+ 

300+ 

47 

48 

83 

40 

32 

104 

111 

72 

225 

CDR 

4000+ 

300  + 

.65 

.86 

.83 

.61 

.60 

.55 

.71 

.60 

.61 

V..   RESULTS  OF  PROGRAM  RUNS 

A.         AVERAGE  PILOT  AS  STUDENT 

For  the  case  of  the  average  pilot  for  the  entire  sample  as  the 
student,  a  successful  evaluation  of  the  situations  occurred  early  in  the 
runs,  i.e.  ,  within  the  first  five  trials  for  all  instructors.    As  can  be 
seen  in  Table  VIII,  the  average  pilot  instructing  the  average  pilot  had 
the  highest  overall  average  of  correct  decisions:     85%  for  the  primary 
tactical  decision,   72%  for  the  secondary  enemy  kill  decision  and  94% 
for  the  secondary  friendly  kill  decision.     Naturally,  the  student's  ending 
coefficients  were  the  same  as  the  student's  starting  coefficients  since 
they  are  identical  to  the  instructor's.    The  extremely  high  percentage  of 
secondary  friendly  kill  decisions  is  puzzling  in  light  of  the  low  corre- 
lation coefficient  of  .4015  between  the  predictor  eguation  with  actual 
secondary  kill  decisions  made.    Varying  the  random  number  seed  changed 
that  percentage  negligibly.     An  explanation  may  lie  in  the  fact  that  the 
rounding  of  the  program  decision  equations  to  the  nearest  integer  improved 
the  accuracy  of  the  prediction  with  respect  to  the  correlation  observed. 
For  example,  a  predicted  value  of  1.4  9  friendly  kills  would  be  rounded 
to  one  friendly  kill  by  the  artificial  intelligence  program.    Assuming  the 
actual  friendly  kill  decision  to  be  one  friendly  kill,  the  artificial  intel- 
ligence program  would  show  a  higher  correlation  than  the  straight 
predictor  equations  . 
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On  the  assumption  that  a  student,  upon  reaching  90%  of  his  final 
performance,  is  near  matriculation,  the  time  for  his  running  percentage 
of  correct  decisions  to  reach  90%  of  his  final  percentage  after  his  first 
correct  decision  was  calculated.    The  running  percentage  of  correct 
decisions  reached  90%  of  the  final  percentage  within  five  trials  for  the 
primary  tactical  decision,  five  correct  primary  decisions  (a  correct 
primary  decision  corresponds  to  a  trial  for  the  secondary  decision)  for 
the  secondary  friendly  kill  decision  and  19  correct  primary  decisions  for 
the  secondary  enemy  kill  decision. 

The  Commander  instructor,  as  can  be  seen  in  Table  VIII,  whose 
predictor  equations  differed  the  most  from  the  average  pilot's,  showed 
the  lowest  final  percentage  of  successful  decisions  except  in  the  case 
of  the  secondary  friendly  kill  decision;  67%  for  the  primary  tactical 
decision,   63%  for  the  secondary  enemy  kill  decision  and  71%  for  the 
secondary  friendly  kill  decision.    The  high  friendly  kill  decision  per- 
centage may  be  a  reflection  of  a  rank-friendly  kill  correlation  witnessed 
in  the  data  analysis   (See  Table  II)  .    The  running  percentage  of  correct 
decisions  reached  90%  of  the  final  percentage  within  62  trials  for  the 
primary  decision,  3  9  correct  primary  decisions  for  the  enemy  kills  and 
57  correct  primary  decisions  for  the  friendly  kills.     These  longer  learning 
times  are  felt  to  be  reflections  of  the  instructor  and  student  coefficient 
differences . 
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The  4000-plus  hour  instructor,  as  can  be  seen  in  Table  VIII, 
closely  matched  the  average  pilot  instructor  in  final  percentage  of 
primary  tactical  decisions  with  87%  and  neared  the  Commander  instructor 
in  final  percentage  of  secondary  enemy  kill  decisions  with  60%.    The 
4  000-plus  instructor  displayed  the  lowest  secondary  friendly  kill  decisions 
with  62%.    The  time  for  running  percentages  to  reach  90%  of  the  final 
percentages  for  this  run  were  moderately  low  at  40  trials  for  the  primary 
decision,  and  eight  correct  primary  decisions  for  both  secondary  decisions. 
The  low  number  of  primary  decisions  correct  needed  to  reach  90%  of  the 
secondary  percentages  is  a  reflection  of  the  lower  final  percentages. 

The  300-plus  mission  instructor,  as  shown  in  Table  VIII  yielded 
final  percentages  of  83%,  55%  and  64%  for  the  primary  and  secondary 
enemy  and  friendly  kill  decisions,  respectively.     Running  percentages 
reached  90%  of  final  percentages  within  96  trials  for  primary  decisions, 
73  correct  primary  decisions  for  the  secondary  enemy  kill  decision  and 
95  for  the  friendly  kill  decision.    These  longer  learning  times  are  felt 
to  be  a  result  of  the  modification  factor  (Z3)  rather  than  a  reflection  of 
lower  percentages  or  instructor  student  differences.    Although  no  such 
run  was  made,  the  panel  of  aviators  felt  that  a  changed  random  number 
seed  would  show  reduced  learning  times. 

However,  as  can  be  seen  on  the  graphs  of  total  decisions  right 
versus  total  trials  or  total  primary  decisions  correct  (See  Figures  4-15), 
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the  results  are  similar  and  nearly  linear  for  all  cases.    The  panel  of 
aviators  felt  this  pattern  to  be  consistent  with  their  own  experience, 
i.e.  ,  as  a  result  of  standardization  of  procedures  and  training,  different 
instructors  teaching  similar  pilots  would  have,  for  the  most  part,  similar 
results..    They  also  considered  the  student,  the  average  pilot  from  the 
sample,  as  more  experienced  than  the  actual  student  encountered  in  a 
flight  syllabus.     However,  in  a  refresher  or  replacement  training  situation 
where  the  student  is  experienced,   such  rapid  attainment  of  consistent 
results  as  witnessed  in  the  graphs  is  felt  by  the  panel  to  be  typical. 

B.         SYNTHETIC  BEGINNER  AS  STUDENT 

The  synthetic  beginner  student,  with  all  starting  coefficients  set 
to  zero  displayed  a  radically  different  behavior  than  that  of  the  average 
pilot  (See  Table  VIII)  .  At  the  end  of  900  trials,  all  the  running  per- 
centages for  the  last  100  decisions  were  equal  to  the  final  percentages 
witnessed  when  the  average  pilot  was  the  student.  Inspection  of  the 
runs  revealed  the  interesting  phenomenon  of  long  error  periods  followed 
by  rapid  learning  to  a  steady  percentage. 

With  the  average  pilot  as  instructor,  the  first  correct  primary 
tactical  decision  was  reached  on  the  200th  trial.    The  first  correct 
secondary  enemy  kill  decision  was  not  reached  until  after  12  9  correct 
primary  decisions  had  been  made,   the  first  correct  secondary  friendly 
kill  decision  was  not  reached  until  after  157  correct  primary  decisions. 
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Figure  5 
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Figure  6 
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Figure  7 
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Figure  9 
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Figure  10 
AVERAGE  STUDENT-4  000  HOUR  INSTRUCTOR 


Go°r  PRIMAPxY     KIQHT      VERSUS    TRIALS 


PRIMARY     %HT 


SCO 


400 


SQO 


eoo 


Figure  11 
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Figure  12 
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Figure  13 
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Figure  14 
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Figure  15 


AVERAGE  STUDENT -3 00  MISSION  INSTRUCTOR 


600  r 


SECONDARY      FRIENDLY      KILLS      RIQHT 

VERSUS 
PRIMARY      R1QHT 


lOO  2,00  300 

SECOMDARY  FRIETNDLY 


400 


TOO 

R I  G,  HT 


u<;C> 


63 


It  took  40  additional  trials  for  the  correct  primary  decision  percentage 
to  reach  90%  of  the  final  percentage.    The  secondary  enemy  and  friendly 
kill  decision  percentages  took  13  7  and  5  6  additional  correct  primary 
decisions,  respectively,  to  reach  90%  of  their  final  percentages. 

With  the  Commander  instructor,  the  first  correct  primary  decision 
was  attained  after  221  trials,  the  first  correct  secondary  enemy  kill 
and  friendly  kill  decisions  after  194  and  177  correct  primary  decisions, 
respectively.    After  4  7  additional  trials,  the  correct  primary  decision 
percentage  reached  9  0%  of  the  final.    The  secondary  enemy  and  friendly 
kill  decision  percentages  reached  90%  of  the  final  percentages  after  4  0 
and  111  additional  correct  primary  decisions,  respectively. 

With  the  4000-plus  hour  instructor,  the  first  correct  primary 
decision  was  reached  after  199  trials,  the  first  correct  secondary  enemy 
and  friendly  kill  decisions  coming  after  172  and  161  correct  primary 
decisions,  respectively.    Forty -eight  additional  trials  were  needed  to 
attain  90%  of  the  final  correct  primary  tactical  decision  percentage,  32 
and  72  additional  correct  primary  decisions  were  needed  to  attain  90% 
of  the  final  correct  secondary  enemy  and  friendly  decision  percentages. 

The  300-plus  mission  instructor  used  206  trials  before  the  first 
correct  primary  tactical  decision  was  reached,   14  9  and  177  correct 
primary  decisions  before  the  first  correct  secondary  enemy  and  friendly 
kill  decisions,  respectively.     Eighty-three  additional  trials  were  needed 
to  attain  90%  of  the  final  correct  primary  tactical  decision  percentage, 
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Figure  17 
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Figure  18 
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Figure  19 
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Figure  21 
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Figure  22 
BEGINNING  STUDENT-4000  HOUR  INSTRUCTOR 
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Figure  23 
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Figure  24 
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Figure  2  5 
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Figure  2  7 
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104  and  225  additional  correct  primary  decisions  to  reach  90%  ofthe 
first  correct  secondary  enemy  and  friendly  kill  percentages,  respectively. 

In  comparison  with  the  runs  and  graphs  for  the  average  pilot  as 
student,  three  significant  points  stand  out  (See  Figures  4-15,   16-27 
and  Table  VIII): 

(1)  The  beginner  requires  a  lengthy  time  to  attain  even  one 
correct  decision,  no  matter  what  decision  is  being  considered 

(2)  After  a  correct  decision,  the  beginner  in  general,  needs  a 
longer  learning  time  to  reach  the  average  pilot's  degree  of 
sophistication. 

(3)  Once  this  degree  of  sophistication  is  reached,  the  beginner 
and  average  pilot  are  nearly  identical  in  behavior. 

In  essence,  the  beginner  is,  indeed,  a  beginner.     In  fact,  the 
beginner  could  even  be  considered  the  average  pilot  three  hundred  trials 
younger.    The  panel  of  aviators  considered  the  beginner  as  analogous  to 
the  beginning  aviation  student:    the  long,  flat,  error  period  being  as 
in  his  ground  and  basic  schooling  where  the  basic  ideas  and  guidance 
are  implanted;  the  learning  period  as  the  time  when  a  student's  motor 
skills  are  honed  to  the  point  that  unencumbered  mental  integration  of  the 
entire  flight  situation  is  developed  -  the  transition  from  flight  student 
to  a  full  fledged  Naval  aviator;  and  the  more  constant,  steady  period 
as  the  reliable  activity  of  an  experienced  pilot. 

As  can  be  seen  in  Table  VI,  discrepancies  do  appear  between 
instruction  as  in  the  average  pilot  student  runs.    Whether  these  dis- 
crepancies are  significant  or  not  is  a  matter  of  conjecture.     In  light 
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of  the  data  analysis  showing  little  correlation  of  biographical  features 
with  decisions  reached,  this  question  of  significant  instructor  differences 
must  remain  unanswered  unless  further  research  is  conducted. 

C        THE  ARTIFICIAL  INTELLIGENCE  QUESTION 

Does  this  program  and  its  resulting  runs  fall  into  the  classification 
of  artificial  intelligence?    Taking  the  program  in  its  entirety,  one  finds 
that  it  does  analyze  its  own  performances,  does  diagnose  its  failures 
and  does  make  changes  that  enhance  its  future  effectiveness  and  per- 
formance.    This  behavior,  if  witnessed  in  a  human  being,  would  be 
called  intelligent  behavior.    Therefore,  by  previously  cited  criteria 
[8,    11]  this  program  does,  indeed,  fall  into  the  classification  of 
artificial  intelligence. 
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VI..    CONCLUSION 

In  summation,  the  low  correlations  between  the  biographical 
factors  of  rank,  hours  and  combat  missions  and  the  decisions  made  in 
the  evaluation  of  an  air-to-air  combat  situation. is  disconcerting.    The 
explanation  that  the  standardization  of  training  and  flight  procedures 
may  account  for  this  lack  of  relationship  seems  of  doubtful  validity 
to  the  author.    The  one  area  showing  any  hint  of  a  relationship  is  rank 
and  the  prediction  of  friendly  kills.     Still,  the  coefficient  of  correlation 
is  weak  (See  Table  II)  . 

The  multiple  correlations  between  the  decisions  and  situational 
variables  are  moderately  low  to  moderately  high,  varying  from  .3378  to 
.8193.     The  lack  of  any  high  correlation  coefficient  values  may  be  due 
to  randomness  in  the  primary  and  secondary  decision  making  among  the 
pilots.     It  may  also  be  due  to  the  failure  of  the  multiple  regression  to 
capture  the  decision  making  policy  either  due  to  missing  variables  or 
a  failure  to  capture  a  non-linear  relationship  with  a  linear  function. 
Still,  in  the  case  of  the  average  pilot,  the  equations  produced  from  the 
multiple  regression  predicted  757  correct  primary  decisions  out  of  900, 
548  correct  enemy  kill  decisions  out  of  757,  and  714  correct  friendly 
kill  decisions  out  of  757.     This  indicates  a  rather  adequate  capturing 
of  the  average  pilot's  decision  policy.    The  relatively  low  correlation 
statistics  may,  in  this  light,  be  misleading. 
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On  the  average,  the  variables  most  influencing  a  pilot  in  reaching 
a  tactical  decision  of  the  types  shown  in  Figures  I  and  II  appear  to  be 
MPG  and  danger,  the  least  important  being  fuel  remaining  and  rules  of 
engagement  (See  Table  VI).     The  fact  that  fuel  remaining  is  considered 
least  important  while  MPG,  a  function  of  fuel  remaining,  is  considered 
most  important  indicates  the  necessity  of  considering  MPG  as  a  separate, 
though  not  necessarily  independent,  function  of  range  and  fuel  remaining. 

A  study  by  Rigney  and  De  Bow  [6]  has  shown  that  63  Combat  Infor- 
mation Center  watch  officers  considered  range  and  enemy  course  (heading) 
as  the  most  important  air  raid  threat  variables.    Although  the  watch 
officer's  situation  is  radically  different  from  that  of  the  pilot,  the  fact 
that  range  is  an  argument  of  the  MPG  function  and  course  is  an  argument 
of  the  danger  function  is  interesting,  especially  in  light  of  the  fact  that 
the  watch  officer  does  not  consider  fuel. 

The  most  important  variables  considered  in  making  the  secondary 
enemy  kill  decisions  are  the  tactical  decision  reached,  fuel-remaining, 
and  rules  of  engagement  (See  Table  VI) .     The  least  important  are  MPG 
and  the  number  of  enemy.    An  interesting  reversal  of  importance  can  be 
witnessed  in  MPG  being  considered  most  important  in  reaching  a  tactical 
decision  and  least  important  in  the  secondary  enemy  kill  decision.    Also, 
the  two  least  important  variables  in  the  tactical  decision  are  second  and 
third  in  importance  in  predicting  this  secondary  enemy  kill  decision. 
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Number  of  enemy,  tactical  decision  and  speed  seem  most  important 
in  predicting  pilots'  friendly  kill  expectancies  .     Least  important  in  this 
prediction  are  MPG  and  range.    Again,  in  comparison  with  the  tactical 
decision,   MPG  is  seen  to  switch  roles.    Also  noticeable  is  the  number 
of  enemy's  rise  to  most  important  while,  in  the  enemy  kill  decision,  it  is 
the  next  to  least  important.    The  three  decisions  appear  to  be,  in  fact, 
three  separate  decisions,  the  first  (primary  tactical)  affecting  the  others 
(enemy  kills  and  friendly  kills)  but  neither  of  the  latter  two  affecting 
each  other. 

Looking  at  the  pilots  individually,  and  within  their  biographical 
groups,  differences  do  appear  to  exist  in  how  they  reach  their  tactical 
decisions.     But  these  differences  do  not  appear  to  be  correlated  with 
rank,  hours  or  missions  but  are,  rather,  insignificant  random  pertur- 
bations.   Secondary  decisions  show  little  cross-group  variation  (See 
Table  VI  and  Appendix  G)  . 

The  artificial  intelligence  program  does  appear  to  show  adaptive 
learning  behavior  when  the  program  is  taken  in  its  entirety.    The  behavior 
of  the  program  can  reasonably  be  termed  intelligent.    Therefore,  one 
could  safely  say  that  the  variables  and  factors  influencing  an  air-to-air 
combat  evaluation  can,  for  the  most  part,  be  quantified  and  utilized  in 
an  artificial  intelligence  application  of  air-to-air  combat  decision 
making  policies . 


VII.    CRITICAL  EVALUATION  OF  THE  STUDY 


Several  pertinent  critical  comments  pertaining  to  this  study  are: 

(1)  The  group  of  36  pilots  from  which  data  were  gathered  is  too  small 
to  allow  any  justifiable  generalization  to  Navy  pilots  in  general. 
Consequently,  as  a  result  of  this  small  size,  any  statistics  and 
findings  generated  may  not  be  representative  of  :Navy  pilots  in 
general. 

Granted,  no  generalizations  about  all  Navy  fighter  pilots  can  be 
safely  made  from  these  findings.     An  appreciably  larger  sample 
is  required.     Still,  the  sample  did  cover  2304  separate  evaluations. 
Any  generalizations  drawn  do  apply,  at  least,  to  the  group  and 
situations  studied. 

(2)  The  assumption  that  certain  factors,   such  as  rank,  primary 
decisions,  and  rules  of  engagement,  can  be  quantified  is  gues- 
tionable,  i.e.,  does  a  Commander  possess  five  times  the  amount 
of  rank  as  an  Ensign? 

Granted,  especially  in  the  area  of  attempting  to  quantify  a  variable 
such  as  rules  of  engagement.     Still,  quantifying  rank  and  placing 
the  decisions  on  a  continuum  are  not  necessarily  unrealistic. 
Perhaps  a  better  substitute  for  rank  would  be  lineal  number  and 
the  subjects  should  be  asked  to  give  a  number  value  on  a  decision 
continuum;  the  relationships  between  primary  decisions  as  presented 
not  necessarily  being  linear. 

(3)  The  situational  variables  such  as  bearing,  heading,  fuel  states, 
range  and  number  of  enemy  are  not,  in  reality,  restricted  to  only 
two  states  . 

Granted,  however,  questionnaire  size  limitations  enforces  such 
a  binary  state.     Still,  a  better  data  base  could  result  from  a 
broader  representation  of  the  situational  variables. 

(4)  In  light  of  the  low  biographical-decision  correlation,  how  can  one 
assume  that  expertise  is  proportional  to  a  biographical  attribute? 

The  question  of  expertise,  although  interesting,  is  essentially 
a  moot  one.  The  only  true  criteria  of  expertise  in  the  situation 
would  deal  with  the  results  of  actual  air-to-air  combat.  Since 
the  number  of  people  with  this  kind  of  expertise  is  small  and  not 
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available  to  the  researcher,   "experts"  were  picked  by  the  previously 
described  method..   Whether,  as  a.rcsult  of  learning,  the  student 
has  developed  into  a  better  or  worse  fighter  pilot  cannot  be  deter- 
mined is  irrelevant.    What  is  relevant  is  whether  or  not  the  student 
approaches  the  standards  of  his  declared  "expert"  instructor. 

(5)  It  appears  unrealistic  to  ask  a  pilot  with  no  combat  experience  to 
evaluate  a  combat  situation. 

Although  seemingly  unrealistic  on  the  surface,  the  armed  forces 
do.  require  a  person  with  no  combat  experience  to  make  an  evaluation 
of  an  actual  combat  situation.     However,  the  person  normally  has 
undergone  extensive  and  realistic  simulated  combat  training.    All 
of  the  36  pilots  studied  have  undergone  such  training.    An  interes- 
ting note  is  the  similarity  in  decisions  between  the  no  combat 
group  and  Lieutenant  Commander  group  which  has  an  average 
combat  total  of  over  160  missions   (See  Table  VI  and  Appendix  G) . 

(6)  The  artificial  intelligence  program  is  simply  a  model  of  a  situation- 
instructor-student  interaction. 

Whether  the  artificial  intelligence  program  is  a  model  or  does  not 
detract  from  the  fact  that  it  meets  the  stated  criteria  of  artificial 
intelligence . 

(7)  Certain  factors  may  be  used  for  predictors  several  times  in  the 
same  equation,   such  as  MPG,  range  and  fuel,  raising  the  multiple 
correlation  accordingly. 

Granted,  but  the  apparent  separation  of  seemingly  related  functions 
in  the  ranking  scales   (See  Table  VI  and  Appendix  G)  and  the  exper- 
ience of  the  panel  of  aviators  does  not  make  this  practice  appear 
unrealistic. 

(8)  The  student's  confidence  level  increases  with  success  but  does 
not  decrease  with  failure,  a  situation  that  may  be  unrealistic. 

Granted,  however,  when  a  decreasing  confidence  level  was  imple- 
mented, the  modification  factor  in  the  beginning  stages  grew  so 
large  as  to  cause  erratic  and  rapid  convergence  to  the  instructor's 
coefficients.     Overcoming  this  problem,  however,  would  not  be 
difficult. 
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VIII..    CONTINUATION  OF  THE  STUDY 

More  data  are  desperately  needed  before  any  of  the  results  un- 
covered can  be  safely  generalized  to  Navy  pilots  in  general.  Likewise, 
more  situations  are  needed  for  presentation  to  the  subjects.  A  factor 
indicating  a  consideration  of  the  enemy's  plight  must  also  be  sought. 
For  example,  variables  indicative  of  the  friendly  airplane  pointing  at 
the  enemy  or  number  of  friendlies  could  be  considered. 

The  computer  graphics  field  is  a  likely  method  to  use  in  future 
data  collection  and  in  developing  real  time  interactive  artificial  intel- 
ligence applications.    For  example,  the  cathode  ray  tube  screen  could 
easily  be  configured  to  resemble  a  cluster  of  instruments  and  radar  scope. 
The  situation  would  then  be  presented  on  the  scope  via  simulated  radar 
images,  digital  readings  on  the  instrument  and  appropriate  character 
generation.    Via  random  number  generation  a  multitude  of  situations 
could  be  presented  to  the  subject.    Teletype  or  light  pen  input  would 
increase  the  subject's  interaction  and  speed  of  response,   allowing 
several  hundred  responses  in  the  space  of  an  hour.     Real  time  data 
collection  and  analysis  could  produce  desired  statistics  and  multiple 
regression  equations  on  demand.     In  the  same  manner,  by  using  the 
artificial  intelligence  program  described  in  this  study,  the  human  could 
take  over  the  role  of  instructor  with  appropriate  modifications  for  response 
collection  and  analysis.    By  not  utilizing  the  modification  subroutines  / 
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the  human  could  take  the  place  of  the  student,  his  coefficients  being 
evaluated  and  graded  by  the  instructor,  followed  by  a  response  such 
as  "You  are  not  emphasizing  your  fuel  state  enough"  or  "You  are  neglecting 
your  range  consideration."    The  training,   selection  and  research  possi- 
bilities in  this  area  are  boundless,  limited  only  by  the  researcher's 
imagination.    Through  this  kind  of  work,  the  fighter  pilot  may  become 
less  of  an  enigma,   less  of  a  superman;  more  of  a  human. 


APPENDIX  A 

QUESTIONNAIRE 

1 .  a)         Relative  position  (range  and  bearing)  of  the 

enemy  aircraft  to  the  strike  force. 

b)  Type  enemy  aircraft  suspected  or  known. 

c)  I  cannot  choose  between  them 

2.  a)  Composition  of  enemy  force — number  of  raid(s) 

and  number  of  aircraft  in  each  raid. 

b)  Number  of  assigned  TARCAP  aircraft  available. 

c)  I  cannot  choose  between  them. 

3.  a)         Range  of  enemy  aircraft  to  the  strike  group. 

b)  Speed  (absolute)  of  the  enemy  aircraft. 

c)  I  cannot  choose  between  them. 

4.  a)  Relative  position  (range  and  bearing)  of  the 

enemy  aircraft  to  the  strike  group. 

b)  Range  of  enemy  aircraft  to  the  strike  group. 

c)  I  cannot  choose  between  them. 

5.  a)         Type  enemy  aircraft  suspected  or  known. 

b)  Capabilities  and  systems  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of 
missiles  ,  etc  .) . 

c)  I  cannot  choose  between  them. 

6.  a)  Speed  (absolute)  of  the  enemy  aircraft. 

b)  Altitude  of  the  enemy  aircraft. 

c)  I  cannot  choose  between  them. 
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7.  a)  Anti-aircraft,  including  surface  to  air  mis- 

sile (SAM)  ,  activity  in  your  immediate  area  . 

b)  Capabilities  and  systems  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of 
missiles,  etc .) . 

c)  I  cannot  choose  between  them. 

8.  a)         Altitude  of  the  enemy  aircraft 

b)  Composition  of  enemy  force — number  of  raid(s) 
and  number  of  aircraft  in  each  raid. 

c)  I  cannot  choose  between  them. 

9.  a)  Number  of  assigned  TARCAP  aircraft  available. 

b)  Bearing  of  the  enemy  aircraft  from  yourself. 

c)  I  cannot  choose  between  them. 

10.  a)  Bearing  of  the  enemy  aircraft  from  yourself. 

b)  Anti-aircraft,   including  surface  to  air  mis- 
sile (SAM),  activity  in  your  immediate  area. 

c)  I  cannot  choose  between  them. 

11.  a)  Relative  position  (range  and  bearing)  of  the 

enemy  aircraft  to  the  strike  force. 

b)  Rate  of  closure  of  enemy  aircraft  to  yourself.    » 

c)  I  cannot  choose  between  them. 

12.  a)  Rate  of  closure  of  enemy  aircraft  to  yourself. 

b)  Composition  of  enemy  force — number  of  raid(s) 
and  number  of  aircraft  in  each  raid. 

c)  I  cannot  choose  between  them. 
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13. 

a) 

b) 

c) 

14. 

a) 

b) 

c) 

15. 

a) 

Number  of  assigned  TARCAP  aircraft  available 

Range  of  enemy  aircraft  to  the  strike  force. 

I  cannot  choose  between  them. 

Speed  (absolute)  of  the  enemy  aircraft. 

Relative  position  (range  and  bearing)  of  the 
enemy  aircraft  to  the  strike  force. 

I  cannot  choose  between  them. 

Capabilities  and  systems  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of  mis- 
siles ,   etc .)  . 

b)  Range  of  enemy  aircraft  to  the  strike  force. 

c)  I  cannot  choose  between  them. 

16.       a)  Speed  (absolute)  of  the  enemy  aircraft. 

Type  enemy  aircraft  suspected  or  known. 

I  cannot  choose  between  them. 

Anti-aircraft,  including  surface  to  air  mis- 
sile (SAM),  activity  in  your  immediate  area. 

Altitude  of  the  enemy  aircraft. 

I  cannot  choose  between  them. 

Capabilities  and  systems  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of 
missiles  ,  etc  .)  . 

b)  Altitude  of  the  enemy  aircraft. 

c)  I  cannot  choose  between  them. 


b) 

c) 

7. 

a) 

b) 

c) 

8. 

a) 

88 


19.  a)         Type  of  enemy  aircraft  suspected  or  known. 

b)  Anti-aircraft,  including  surface  to  air  mis- 
sile (SAM),  activity  in  your  immediate  area. 

c)  I  cannot  choose  between  them. 

20.  a)         Relative  position  (range  and  bearing)  of  the 

enemy  aircraft  to  the  strike  force. 

b)  Composition  of  enemy  forces — number  of  raid (s) 
and  number  of  aircraft  in  each  raid. 

c)  I  cannot  choose  between  them. 

21.  a)  Rate  of  closure  of  enemy  aircraft  to  yourself. 

b)  Range  of  enemy  aircraft  to  the  strike  force. 

c)  I  cannot  choose  between  them. 

22.  a)  Number  of  assigned  TARCAP  aircraft  available. 

b)  Relative  position  (range  and  bearing)  of  the 
enemy  aircraft  to  the  strike  force. 

c)  I  cannot  choose  between  them. 

23.  a)  Capabilities  and  systems  operability  of  your 

own  aircraft  (e.g.  ,  fuel  state,  number  of  mis- 
siles ,  etc .)  . 

b)  Speed  (absolute)  of  the  enemy  aircraft. 

c)  I  cannot  choose  between  them. 

24.  a)  Capabilities  and  systems  operability  of  your 

own  aircraft  (e.g.  ,  fuel  state,  number  of  mis- 
siles, etc.) . 

b)  Number  of  assigned  TARCAP  aircraft  available. 

c)  I  cannot  choose  between  them. 
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b) 

c) 

26. 

a) 

b) 

c) 

27. 

a) 

25.       a)  Composition  of  enemy  forces — number  of  raid  (s) 

and  number  of  aircraft  in  each,  raid . 

Bearing  of  the  enemy  aircraft  from  yourself.  . 

I  cannot  choose  between  them. 

Type  enemy  aircraft  suspected  or  known. 

Bearing  of  the  enemy  aircraft  from  yourself. 

I  cannot  choose  between  them. 

Rate  of  closure  of  enemy  aircraft  to  yourself. 

b)  Anti-aircraft,   including  surface  to  air  mis- 
sile (SAM),  activity  in  your  immediate  area. 

c)  I  cannot  choose  between  them. 

28.  a)         Anti-aircraft,  including  surface  to  air  mis- 

sile (SAM),  activity  in  your  immediate  area. 

b)  Range  of  enemy  aircraft  to  the  strike  force. 

c)  I  cannot  choose  between  them. 

29.  a)         Altitude  of  the  enemy  aircraft. 

b)  Type  of  enemy  aircraft  suspected  or  known. 

c)  I  cannot  choose  between  them. 

30.  a)         Altitude  of  the  enemy  aircraft. 

b)  Number  of  assigned  TARCAP  aircraft  available. 

c)  I  cannot  choose  between  them. 
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b) 

c) 

33. 

a) 

b) 

c) 

34. 

a) 

31.  a)  Bearing  of  the  enemy  aircraft  from  yourself.  . 

b)  Capabilities  and  system  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of  mis- 
siles, etc .)  . 

c)  I  cannot  choose  between  them. 

32.  (a)        Composition  of  enemy  force — number  of  raid (s) 

and  number  of  aircraft  in  each  raid. 

Type  of  enemy  aircraft  suspected  or  known. 

I  cannot  choose  between  them. 

Bearing  of  the  enemy  aircraft  from  yourself. 

Rate  of  closure  of  enemy  aircraft  to  yourself. 

I  cannot  choose  between  them. 

Composition  of  enemy  force — number  of  raid(s) 
and  number  of  aircraft  in  each  raid. 

b)  Anti-aircraft,  including  surface  to  air  mis- 
siles (SAM),  activity  in  your  immediate  area. 

c)  I  cannot  choose  between  them. 

35.  a)         Rate  of  closure  of  enemy  aircraft  to  yourself. 

b)  Capabilities  and  system  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of  mis- 
siles ,  etc .)  . 

c)  I  cannot  choose  between  them. 

36.  a)  Number  of  assigned  TARCAP  aircraft  available. 

b)  Speed  (absolute)  of  the  enemy  aircraft. 

c)  I  cannot  choose  between  them. 
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3.7,      a)  Speed  (absolute)  of  the  enemy  aircraft .  . 

b)  Anti-aircraft,  including  surface  to  air  mis- 
siles (SAM),  activity  in  your  immediate  area. 

c)  I  cannot  choose  between  them. 

Range  of  enemy  aircraft  to  the  strike  force. 

Altitude  of  the  enemy  aircraft.. 

I  cannot  choose  between  them. 

Type  enemy  aircraft  suspected  or  known. 

Number  of  assigned  TARCAP  aircraft  available 

I  cannot  choose  between  them. 

Altitude  of  the  enemy  aircraft. 

Bearing  of  the  enemy  aircraft  from  yourself. 

I  cannot  choose  between  them. 

Relative  position  (range  and  bearing)  of  the 
enemy  aircraft  to  the  strike  force. 

b)  Capabilities  and  systems  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of  mis- 
siles, etc .) . 

c)  I  cannot  choose  between  them. 

42.       a)  Speed  (absolute)  of  the  enemy  aircraft. 

b)  Rate  of  closure  of  enemy  aircraft  to  yourself. 

c)  I  cannot  choose  between  them. 


38. 

a) 

b) 

c) 

39. 

a) 

b) 

c) 

40. 

a) 

b) 

c) 

41. 

a) 
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43  ..      a)         Anti-aircraft,  including  surface  to  air  mis- 
sile (SAM) ,  .activity  in  your  immediate  area  . 

Number  of  assigned  TARCAP  aircraft  available . 

I. cannot  choose  between  them. 

Anti-aircraft,  including  surface  to  air  mis- 
sile (SAM),  activity  in  your  immediate  area. 

Relative  position  (range  and  bearing)  of  the 
enemy  aircraft  to  the  strike  force. 

I  cannot  choose  between  them. 

Range  of  enemy  aircraft  to  the  strike  force. 

Bearing  of  the  enemy  aircraft  from  yourself. 

I  cannot  choose  between  them. 

Rate  of  closure  of  enemy  aircraft  to  yourself. 

Altitude  of  the  enemy  aircraft. 

I  cannot  choose  between  them. 

Capabilities  and  systems  operability  of  your 
own  aircraft  (e.g.  ,  fuel  state,  number  of  mis- 
siles ,  etc  .) . 

b)  Composition  of  enemy  force--number  of  raid(s) 
and  number  of  aircraft  in  each  raid. 

c)  I  cannot  choose  between  them. 

48.       a)  Range  of  enemy  aircraft  to  the  strike  force. 

b)  Composition  of  enemy  force — number  of  raid(s) 
and  number  of  aircraft  in  each  raid. 

c)  I  cannot  choose  between  them. 
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b) 

c) 

44. 

a) 

b) 

c) 

45. 

a) 

b) 

c) 

46. 

a) 

b) 

c) 

47. 

a) 

49.  a)  Bearing  of  the  enemy  aircraft  from  yourself. 

b)  Relative  position  (range  and  bearing)  of  the 
enemy  aircraft  to  the  strike  force. 

c)  I  cannot  choose  between  them. 

50.  a)  Bearing  of  the  enemy  aircraft  from  yourself. 

b)  Speed  (absolute)  of  the  enemy  aircraft.. 

c)  I  cannot  choose  between  them. 

51.  a)  Range  of  enemy  aircraft  to  the  strike  force. 

b)  Type  enemy  aircraft  suspected  or  known. 

c)  I  cannot  choose  between  them. 

52.  a)  Type  enemy  aircraft  suspected  or  known. 

b)  Rate  of  closure  of  enemy  aircraft  to  yourself. 

c)  I  cannot  choose  between  them. 

53.  a)         Altitude  of  the  enemy  aircraft. 

b)  Relative  position  (range  and  bearing)  of  the 
enemy  aircraft  to  the  strike  force. 

c)  I  cannot  choose  between  them. 

54.  a)  Composition  of  enemy  force—number  of  raid(s) 

and  number  of  aircraft  in  each  raid. 

b)  Speed  (absolute)  of  the  enemy  aircraft. 

c)  I  cannot  choose  between  them. 

55.  a)  Number  of  assigned  TARCAP  aircraft  available. 

b)  Rate  of  closure  of  enemy  aircraft  to  yourself. 

c)  I  cannot  choose  between  them. 
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APPENDIX  B 


PILOTS  RESPONDING  TO  MAIL-OUT  QUESTIONNAIRE 

Rank  Hours  Missions 


1. 

Commander 

4500 

2. 

Commander 

4000 

3. 

Commander 

3500 

4. 

Commander 

330  0 

5. 

Lieutenant  Comma 

nder 

3450 

6. 

Lieutenant  Comma 

nder 

3200 

7. 

Lieutenant  Commander 

3200 

8. 

Lieutenant  Comma 

nder 

3000 

9. 

Lieutenant  Comma 

nder 

2700 

10. 

Lieutenant  Comma 

nder 

2655 

11. 

Lieutenant  Comma 

nder 

2550 

12. 

Lieutenant  Commander 

2500 

13. 

Lieutenant  Comma 

nder 

1900 

14. 

Lieutenant  Comma 

nder 

1600 

15. 

Lieutenant 

2350 

16. 

Lieutenant 

1800 

17. 

Lieutenant 

1800 

18. 

Lieutenant 

1750 

19. 

Lieutenant 

1700 

20. 

Lieutenant 

1700 

21. 

Lieutenant 

1700 

22. 

Lieutenant 

1500 

23. 

Lieutenant 

1500 

24. 

Lieutenant 

1400 

25. 

Lieutenant 

1400 

26. 

Lieutenant 

1300 

27. 

Lieutenant 

1150 

28. 

Lieutenant 

1100 

29. 

Lieutenant 

1100 

30. 

Lieutenant 

1050 

31. 

Lieutenant 

1000 

32. 

Lieutenant 

850 

33. 

Lieutenant 

750 

34. 

Lieutenant 

750- 

35. 

Lieutenant 

730 

36. 

Lieutenant  (junior 

grade) 

450 

70 

143 

240 

354 

200 

300 

140 

218 

172 

130 

113 

11 

150 

200 

80 

249 

220 

70 

154 

75 

40 

200 

100 

150 

55 

90 

100 

130 

129 

0 

150 

0 

0 

0 

0 

0 
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APPENDIX  C 

PANEL  OF  AVIATORS 

Commander  (selectee)  -  pilot,  attack  and  fighter  type  aircraft  experience- 
two  instructor  tours  in  the  basic  training  command;  one 
instructor  tour  with  an  attack  Replacement  Air  Group;  three 
combat  tours  in  Southeast  Asia;  candidate  for  M.S.  in 
Computer  Science. 

Lieutenant  Commander  -  pilot;  patrol  type  aircraft  experience;  one 

instructor  tour  in  the  basic  training  command;  one  combat 
tour  in  Southeast  Asia;  candidate  for  M.S.  in  Computer 
Science. 

Lieutenant  -  pilot;  attack  type  aircraft  experience;  one  instructor  tour 
in  the  Replacement  Air  Group;  one  combat  tour  in  Southeast 
Asia;  candidate  for  M.S.   in  Computer  Science. 

Lieutenant  -  Naval  Flight  Officer;  experience  as  airborne  intercept 

controller;  one  instructor  tour  in  the  basic  training  command 
teaching  computer  fundamentals;  one  combat  tour  in  South- 
east Asia;  candidate  for  M.S.  in  Computer  Science. 

Lieutenant  -  Naval  Flight  Officer;  experience  as  airborne  intercept 

controller;  one  tour  working  on  airborne  computer  display 
systems;  one  combat  tour  in  Southeast  Asia;  candidate  for 
M.S.  in  Computer  Science. 
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APPENDIX  D 

DISCRIMINANT  ANALYSIS  FOR  GROUPS  DIFFERENTIATED 
BY  BIOGRAPHICAL  DATA 

Significance  Level  of  ..05 

Grouped  by  Rank: 

Chi  square  =  14  .  0 
Degrees  of  Freedom  =  21 

No  significant  differences  between  groups  differentiated 
by  rank . 

Grouped  by  Hours: 

Chi  square  =  24  .0 
Degrees  of  Freedom  =  28 

No  significant  difference  between  groups  differentiated  by 
flight  hours  . 

Grouped  by  Missions: 

Chi  square  =  44  .0 
Degrees  of  Freedom  =  28 

No  significant  difference  between  groups  differentiated 
by  combat  missions  . 
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APPENDIX  E 

SELECTED  PREDTCTOR  EQUATION  COEFFICIENTS 
Primary  Tactical  Decision  —Corresponds  to  Prico  Vector: 

Average  Pilot  CDR  4000+  300+ 


I.. 

1.51835 

1.23987 

1.74982 

1.12484 

z... 

-.00014 

—.00005 

-.00015 

-.00005 

3.. 

.3  910 

.05223 

.03590 

.05204 

4:.. 

..05065 

.03086 

.05952 

.03043 

&:.. 

-.122  3  9 

-.07430 

-,08141 

-.06924 

6. 

.003  52 

.00223 

.00132 

.00440 

7., 

.0002  8 

.00023 

.00016 

.00079 

8. 

8.24161 

5.59970 

4.11829 

-3.22020 

Secondary  Enemy  Kill  Decision  -  Corresponds  to  Secoen  Vector: 
Average  Pilot  CDR  4000+  300+ 


1. 

.82003 

1.20750 

.23976 

1.06406 

2. 

.00034 

.00018 

.00038 

.00024 

3. 

.02213 

.05697 

.05474 

.01154 

4.. 

.03087 

.12087 

-.01666 

.18579 

5. 

.41223 

.20282 

.34197 

.21952 

6. 

-.00281 

-.00306 

-.00242 

-.00253 

7. 

-.00020 

-.00001 

.00013 

-.00008 

8. 

-7.81100 

-13.56913 

27.20380 

11.08119 

9. 

-.43846 

-.66584 

-.22533 

-.57411 

Secondary  Friendly  Kill  Decision  -  Corresponds  to  Secofr  Vector: 
Average  Pilot  CDR  4000+    .  300+ 


1. 

.56915 

.39760 

.74056 

.17621 

2. 

-.00004 

.00012 

.00009 

.00007 

3. 

-.00558 

-.01702 

-.01714 

-.01947 

4. 

.08816 

.18036 

.08989 

.30649 

5. 

-.09740 

-.23837 

-.25172 

-.26590 

6. 

.00059 

.00129 

-.00023 

.00278 

7. 

.00013 

.00019 

.00014 

.00024 

8. 

3.26289 

28.96692 

2  5. 1082  0 

24.35280 

9. 

-.24901 

-.32516 

-.33688 

-.28021 
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APPENDIX  F 
MEAN  AND  STANDARD  DEVIATIONS  OF  DECISIONS  MADE 


Rank  Group 

Primary  D 

CDR 

X 

1.95 

SD 

.82 

LCDR 

X 

1. 36 

SD 

..80 

LT 

X 

2.01 

SD 

.78 

LTJG 

X 

2.16 

SD 

.76 

Hours 

Group 

4000+ 

X 

2.16 

SD 

.80 

4000 

X 

1.83 

SD 

.79 

3000 

X 

1.92 

SD 

.79 

2000 

X 

2.07 

SD 

.96 

1000 

X 

1.99 

SD 

.77 

Mission  Group 

300  + 

X 

1.74 

SD 

.92 

300 

X 

2.02 

SD 

.74 

200 

X 

2.01 

SD 

.80 

100 

X 

2.01 

SD 

.80 

0 

X 

1.97 

SD 

.75 

X    =    arithmetic  mean 
SD  =    standard  deviation 


Enemy  Kills  Friendly  Kills 

1.40  .61 

1.09  1.01 

.93  .32 

.99  .64 

1.10  .18 

1.00  .48 

.70  .08 

.75  .27 


.83  .41 

.78  .55 

1.44  .51 


r 
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.96  .26 

.94  .61 

1.04  .16 

1.06  .45 

1.00  .26 

.84  .55 


1.56  .88 

1.02  1.28 

1.10  .19 

1.08  .53 

.96  .23 

1.01  .53 

1.10  .23 

1.04  .57 

1.07  .24 

.85  .52 
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APPENDIX  G 


VARIABLE  RANKINGS  FOR  HOUR  AND  MISSION  GROUPS 


HOUR  GROUP: 

Primary  Tactical  Decision 


4D00+ 

3000-4000 

2000-3000 

1000-2000 

0-1000 

MPG 

1: 

5: 

7 

1 

7 

Danger 

5 

2. 

2 

3 

3 

Speed 

3 

3 

3 

4 

2 

Range 

4 

i: 

1 

6 

1 

Number  of  Enemy 

2 

4 

4 

5 

5 

Rules  of 

Engagement 

7 

6 

6 

7 

6 

Fuel  Remaining 

6 

7" 

5 

2 

4 

Secondary  Enemy  Kill  Decision 


4000+ 

3000-4000 

2000-3000 

1000-2000 

0-1000 

IDECIS 

3 

1 

1 

2 

1 

Fuel  Remaining 

1 

4 

4 

1 

2 

Rules  of 

Engagement 

4 

5 

2 

3 

4 

Danger 

5 

3 

3 

4 

5 

Speed 

6 

6 

5 

5 

3 

Range 

2 

7 

6 

6 

- 

Number  of  Enemy 

7 

2 

7 

7 

7 

MPG 

8 

8 

- 

8 

6 

Secondary  Friendly  Kill  Decision 


4000+ 

3000-4000 

2000-3000 

1000-2000 

0-1000 

Number  of  Enemy 

2 

1 

2 

1 

1 

IDECIS 

1 

2 

1 

2 

2 

Speed 

4 

4 

3 

3 

3 

Rules  of 

Engagement 

3 

5 

6 

7 

5 

Fuel  Remaining 

7 

6 

5 

8 

4 

Danger 

8 

3 

7 

6 

6 

Range 

6 

7 

4 

4 

7 

MPG 

6 

8 

- 

5 

8 
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APPENDIX  G.  (continued) 

VARIABLE  RANKINGS  FOR  HOUR  AND  MISSION  GROUPS 

MISSION  GROUP: 

Primary  Tactical  Decision 


300+ 

200-300 

100-200 

1-100 

0 

MPG 

7 

i: 

1 

1 

7 

Danger 

3 

2. 

2. 

2 

2 

Speed 

i: 

3 

3 

5 

3 

Range 

2 

4 

5 

4 

1 

Number  c 

f  Enemy 

4 

5r 

4 

3 

5 

Rules  of 

Engagement 

5 

7 

6 

6 

6 

Fuel  Remaining 

6 

6 

7 

7 

4 

Secondary  Enemy  Kill  Decision 


300+ 

200-300 

100-200 

1-100 

0 

IDECIS 

1 

1 

1 

1 

1 

Fuel  Remaining 

4 

2 

2 

2 

2 

Rules  of 

Engagement 

6 

4 

3 

3 

4 

Danger 

5 

3 

4 

4 

5 

Speed 

7 

7 

6 

5 

3 

Range 

3 

6 

5 

6 

6 

Number  of  Enemy 

2 

5 

7 

7 

7 

MPG 

8 

8 

-- 

8 

- 

Secondary  Friendly  Kill  Decision 


300+ 


Number  of  Enemy 

1 

IDECIS 

5 

Speed 

6 

Rules  of 

Engagement 

3 

Fuel  Remaining 

8 

Danger 

4 

Range 

2 

MPG 

7 

200-300 

100-200 

2 

1 

1 

2 

7 

3 

6 

6 

4 

8 

3 

7 

1-100 
2 
1 
3 


4 
5 

f» 
8 
7 


1 
2 
3 

4 
5 
6 
7 

8 
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APPENDIX  H 

MULTIPLE  CORRELATIONS  OF  PREDICTOR  EQUATIONS 
WITH  PREDICTED  DECISION 


Primary  Tactical 

Decision 

Average  Pilot 

.6032 

CDR 

.5794 

LCDR 

.6236 

LT 

.6101 

LT(jg) 

.7568 

Hours 

4000+ 

.4587 

3000-4000 

.6775 

2000-3000 

.6249 

1000-2000 

.6816 

0-1000 

.6062 

Missions 

300  + 

.7459 

200-300 

.6402 

100-200 

.5778 

1-100 

.6309 

0 

.6213 

Secondary  Enemy     Secondary  Friendly 
Kill  Decision  Kill  Decision 


6027 
5859 
5654 
6527 
8193 


.7930 
.5638 
.5988 
.6755 
.6190 


.7052 
.5748 
.5915 
.6677 
.6143 


.4015 
.4391 
.5299 
.3732 
.5284 


.5840 
.4303 
.5217 
.3378 
.4442 


.5128 
.4524 
.3397 
.5309 
.4499 
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